Sperm tracking and automatic extraction method based on machine learning

Through the method based on YOLO-V4 and deep reinforcement learning, the problems of low monitoring accuracy and low efficiency in sperm monitoring and extraction are solved, efficient and automated extraction of sperm is achieved, and the accuracy and intelligence of sperm monitoring and extraction are improved.

CN114241006BActive Publication Date: 2024-12-03YANGTZE DELTA REGION INST (QUZHOU) UNIV OF ELECTRONIC SCI & TECH OF CHINA

Patent Information

Application Number
CN202111560295.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-20
Publication Date
2024-12-03
Estimated Expiration
2041-12-20

AI Technical Summary

Technical Problem

The prior art has problems in sperm monitoring and extraction with low monitoring accuracy, slow speed and relying on manual control, resulting in low efficiency and unpredictable extraction effects.

Method used

Using the multi-objective sperm real-time tracking method based on YOLO-V4 combined with deep reinforcement learning, sperm position is detected through the YOLO-V4 network, the KCF tracker corrects position error, the Kalman filter estimates sperm position, and uses deep reinforcement to Learn to optimize the control strategy of extracting needles to achieve automatic sperm extraction.

Benefits of technology

It improves the accuracy and speed of sperm monitoring, realizes automatic sperm extraction, and improves the intelligence and efficiency of sperm extraction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114241006B_ABST
    Figure CN114241006B_ABST
Patent Text Reader

Abstract

The present invention belongs to the technical field of sperm monitoring and automatic extraction, and provides a sperm tracking and automatic extraction method based on machine learning. The purpose is to solve the problem that the current main sperm monitoring methods generally have low monitoring accuracy and slow monitoring speed, which is not conducive to real-time tracking of multiple sperm targets. The main scheme includes obtaining real-time image data of sperm samples; realizing multi-sperm tracking through YOLO-V4, KCF tracker, and Kalman filter; calculating the speed of sperm according to the tracking results, and determining the target sperm according to the speed; using YOLO-V4 to detect the position of the extraction needle in the image; constructing the observation state of the extraction needle controller according to the position of the extraction needle and the tracking position of the sperm; inputting the observation state into the actor network in the DDPG deep reinforcement learning model to obtain the control quantity. The control quantity output by the actor network is input into the extraction needle controller, and the control quantity is converted into a control signal executable by the controller. The controller executes the behavior corresponding to the control quantity according to the control signal.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of sperm monitoring and automatic extraction, and relates to a multi-target sperm real-time tracking method based on deep learning and a target sperm automatic extraction method based on deep reinforcement learning. Background Art

[0002] Human semen samples carry a variety of information, and efficient and accurate sperm extraction is extremely important for sperm medical research. With the help of a microscope, fast-moving sperm can be observed, but because sperm is moving, it is difficult for staff to track the target sperm to be extracted, which brings great difficulties to sperm extraction. In addition, even experienced professionals may have unstable operations, which may cause target sperm tracking to be lost or extraction to fail. Designing a completely autonomous multi-sperm tracking, automatic target determination, and automatic target extraction technology is of great significance to sperm extraction, and this technology is expected to improve the efficiency of future reproductive medical treatment.

[0003] At present, there is still a lack of technology for real-time tracking and monitoring of sperm and online extraction of target sperm, which is also the original intention of the relevant technology of the present invention.

[0004] The Chinese invention application with publication number CN111563550A discloses a sperm morphology detection method and device based on image technology. The detection method involves computer vision, specifically: identifying any independent sperm in the semen image to be tested; obtaining the contour of the key part of the independent sperm; determining the central coordinates of the key part of the independent sperm from the contour of the key part of the independent sperm; extracting the contour of the independent sperm based on the semen image to be tested and the central coordinates of the key part of the independent sperm; and monitoring the morphology of the independent sperm according to the contour of the independent sperm.

[0005] The Chinese invention application with publication number CN112150415A discloses a multi-target sperm tracking scheme based on SSD and KCF tracker. The method includes the following steps: (1) obtaining real-time image data of sperm samples; (2) sampling feature maps of multi-layer sperm sample images through the SSD network model, extracting features using CNN, classification and regression; (3) transmitting data to the KCF tracker; (4) for the part where the SSD monitoring sperm target is lost, using the KCF tracker to monitor the target in motion and update the result; (5) using the Kalman filter to allocate the tracker; (6) matching the position monitored by the SSD with the distance between the target estimated by the Kalman filter and the KCF tracker to obtain real-time monitoring data of multi-target sperm. Although the SSD deep network model used in this method has a good effect in sperm detection in samples, due to the constraints of the SSD model itself, the real-time performance of sperm detection speed is not ideal.

[0006] In general, the existing technology has the following main defects:

[0007] (1) The main sperm monitoring methods currently available generally have low monitoring accuracy and slow monitoring speed, which is not conducive to real-time tracking of multiple sperm targets.

[0008] (2) The current sperm extraction method relies on manual control of the extractor, which is inefficient and difficult to achieve the expected extraction effect. In addition, fully autonomous sperm extraction technology still needs to be studied. Summary of the invention

[0009] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a multi-target sperm real-time tracking method based on YOLO-V4 and an automatic sperm extraction method based on deep reinforcement learning. In the future, it can be integrated into the application of computer automatic analysis and sperm automatic extraction, so that high-quality sperm can be extracted more conveniently. In order to more conveniently describe the sperm extraction method based on deep reinforcement learning, the present invention uses s k Indicates the current state of the extraction needle controller, a means in state s k The action taken by the next controller, s k+1 Indicates the state of the next control moment, s k-1 represents the state at the last control moment, r represents the controller in state s k The reward obtained by executing action a. The object of the present invention is achieved through the following technical solutions.

[0010] A sperm tracking and automatic extraction method based on machine learning comprises the following steps:

[0011] (1) Obtain real-time image data of sperm samples.

[0012] (2) The sperm features are learned through the YOLO-V4 deep network model. After learning, YOLO-V4 can well complete the detection of sperm, obtain the position of sperm in the image, and transmit the data to the KCF tracker. The YOLO-V4 network model is a target detection method based on deep learning that can realize multi-target parallel inspection. Compared with the YOLO-V3 network model, it has better effect in small target inspection. This method can directly obtain the position of different targets through the deep learning model. This method requires a large number of samples to train the deep network model before use. After the network model is trained with enough samples, the target detection will have higher accuracy and the network model will have better adaptability to different environments.

[0013] (3) For sperm that are missed by the YOLO-V4 network model, the KCF tracker is used to detect the position of the missed sperm by finding the correlation between the target in the previous frame and the current frame image. The KCF tracker detects the target by finding the correlation between the target in the previous frame and the current frame image. By continuously using this method, the target can be tracked. This method has a fast detection speed and can reach more than 150 frames. The KCF algorithm maps the ridge regression in the linear space to the nonlinear space through the kernel function. By solving a dual problem and some common constraints in the nonlinear space, the calculation can also be simplified by using the circulant matrix Fourier space diagonalization. In addition, the KCF algorithm improves the grayscale features that could only use a single channel to now use multi-channel HOG features or other features.

[0014] (4) Since the sperm position detected by the KCF tracker through correlation has a large error, the Kalman filter is used to estimate the position of the tracked target in the current image frame based on past position information to correct the data obtained by the KCF tracker itself.

[0015] (5) According to the distance between the sperm position monitored by the YOLO-V4 network model and the sperm position corrected by the Kalman filter, the KCF tracker is updated based on the minimum distance principle.

[0016] (6) Based on the position information of each frame of the sperm tracker, the displacement of the sperm on the image during adjacent frames is calculated to obtain the speed of sperm movement.

[0017] (7) Calculate the average movement speed of sperm within 10 seconds of observation, and select the sperm with the largest average speed as the target for extraction. The tracking position of the target sperm is denoted by x. k express.

[0018] (8) Use the trained YOLO-V4 network to detect the position of sperm and sperm extraction needle in the current image, and use G k It means that detection is performed every 4 frames.

[0019] (9) Using the currently detected position G of the extraction needle and the target sperm k 、x k and the sperm position x of the last test k-1 Construct observation state s k =(G k ,x k-1 ,x k ). k It contains the position of the sperm at the previous moment and the current moment. This construction method can effectively extract the historical movement information of the target sperm.

[0020] (10) Set the state s k Input the trained deep reinforcement learning network model, and the neural network model outputs the optimal sperm extraction needle control value u = (u1, u2, u3) according to the characteristics of the input state. During the reinforcement learning training process, the extraction needle controller learns the optimal extraction strategy through continuous trial and error.

[0021] (11) The control quantity u is transmitted to the extraction needle controller. The controller converts the three control quantities u1, u2, and u3 into control signals recognizable by the stepper motor, and then controls the three stepper motors respectively to adjust the extraction needle in the x, y, and z directions.

[0022] (12) If the extraction needle moves to the extractable range of the target sperm, the controller controls the extraction needle to perform the extraction operation. If the target sperm is extracted, the control ends. If the extraction needle does not reach the extractable range of the target sperm or the extraction needle does not extract the target sperm after performing the extraction operation, go to step (8).

[0023] In the method of sperm tracking and automatic extraction based on machine learning provided by the present invention, the present invention realizes the detection of sperm and sperm extractor based on the YOLO-V4 network model, realizes the tracking of sperm based on the KCF tracker and Kalman filtering method, and uses a deep reinforcement learning method based on the detection and tracking results to optimize the behavior strategy of the sperm extraction needle, so that the extraction needle can extract the target sperm with the optimal strategy. The present invention constructs the observation state in reinforcement learning according to the tracking and detection results of sperm and extraction needle, and constructs a state containing current information and historical information. In the strong chemical model, the DDPG algorithm adapted to continuous space control is used, and a lightweight BP neural network structure is designed. The complexity of this network can meet the requirements of real-time control of the extraction needle. The reinforcement learning neural network directly outputs the control amount of the extraction needle according to the input observation state, thereby improving the intelligence of sperm extraction.

[0024] Furthermore, the method also includes the following steps: after obtaining the real-time image data of the sperm sample in step (1), the image sample set is classified, and some samples are used as training sets and validation sets to train the YOLO-V4 network model; and some samples are used as test sets to test the trained YOLO-V4 network model.

[0025] Furthermore, step (10) also includes a training method for the deep reinforcement learning model, and the steps of the training method are as follows:

[0026] (1) According to steps (7), (8), and (9) of the sperm tracking and automatic extraction method based on machine learning, the initial observation state s0 is calculated, and based on the initial observation state s0, the initial behavior a is calculated using the created BP neural network, and the behavior a is the control quantity u;

[0027] (2) The controller of the extraction needle executes behavior a. The extraction needle controller converts the control quantity u output by the network into an electronic step control signal and completes the position adjustment of the extraction needle;

[0028] (3) Calculate the latest observed state s of the target sperm k ;

[0029] (4) Calculate the reward value r of the previous behavior based on the reward function k ;

[0030] (5)Tuple (s k-1 ,s k ,a,r k ) is stored in a sample pool;

[0031] (6) Updates k-1 ,s k-1 =s k , and calculate the behavior a of the new state through the BP neural network;

[0032] (7) determining whether the extraction needle has moved to the extraction range, and if so, executing the extraction operation;

[0033] (8) Randomly extract a fixed number of sample tuples from the sample pool for training the BP neural network;

[0034] (9) If the number of times the extraction needle is controlled exceeds the preset number, go to step (10), otherwise go to step (2);

[0035] (10) If the number of training rounds does not reach the preset number, go to step (1); otherwise, end the training and save the trained neural network model.

[0036] Furthermore, in step (4) of the deep reinforcement learning training process, the present invention designs a reward function for target sperm extraction, which is designed based on the distance between the extraction needle and the target sperm and whether the extraction is successful. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the process of the present invention.

[0038] Figure 2 Schematic diagram of the training process of deep reinforcement learning in step 10 of the present invention. DETAILED DESCRIPTION

[0039] The present invention is described in detail below with reference to the accompanying drawings.

[0040] (I) Obtaining training sample image data of sperm and extractor

[0041] Semen samples were obtained and incubated at 37°C for 30 minutes to liquefy the semen. Before capacitation, semen samples were centrifuged at 300 × g for 20 minutes. 40 and 80 (Nidacon, Sweden) for purification. The supernatant was removed, the sperm pellet was resuspended and The cells were centrifuged again 38 times in washing buffer (Nidacon, Sweden) (500×g, 10 min).

[0042] The extraction needle is inserted into the semen sample, and the extraction needle controller is used to adjust the random movement of the extraction needle. The video of the movement of the extraction needle in the semen sample is recorded with a camera, and the video frame rate is 20FPS. 6,000 sample images are generated using video images and classified. 81% of the samples are classified as training samples, 9% of the samples are classified as verification samples, and 10% of the samples are test samples. These samples are used to train the YOLO-V4 network to obtain a trained network model.

[0043] (ii) Use the YOLO-V4 network model to detect sperm and extractors.

[0044] Compared with other network models, the YOLO-V4 network model has the advantages of fast detection speed and good detection effect on small targets. It is very suitable for sperm detection and tracking involved in the present invention. In the whole process, only the image to be detected needs to be input to quickly obtain the location of sperm. In addition, the network also has a good detection effect on simple structure targets such as sperm extraction needles.

[0045] In some specific embodiments, the image in the video stream is reshaped into a 1200×936 image and input into the trained YOLO-V4 network model for detection, and the detection can obtain the position of the sperm and the extraction needle in the image. The YOLO-V4 network model outputs the positions of the four corners of the box where the target is located. When constructing the state of reinforcement learning, the present invention uses the center position of the box to represent the position of the target sperm and the extraction needle.

[0046] (III) Using KCF for multi-sperm tracking

[0047] After obtaining the sperm target frame based on the YOLO-V4 network model, the KCF tracker is used to track the sperm target frame.

[0048] The KCF algorithm maps the ridge regression of the linear space to the nonlinear space through the kernel function. In the nonlinear space, by solving a dual problem and some common constraints, the calculation can be simplified by using the diagonalization of the circulant matrix Fourier space. In addition, the KCF algorithm improves the grayscale feature that could only use a single channel to the HOG feature or other features that can now use multiple channels. The present invention uses the HOG feature to characterize the characteristics of sperm in the image.

[0049] The Fourier transform of the α parameter of the vector tracker can be trained through the image of the current frame It can be expressed as

[0050]

[0051] K XX =φ(X)φ(X) T (2)

[0052] Among them, K XX is a circulant kernel matrix, and the superscript XX represents the correlation between the generated samples in the circulant sample matrix X generated by the training sample vector x. K XX Fourier transform, λ is the regularization parameter, φ(x) is the column vector, φ(g) is the kernel function, (g)* represents the conjugate operation, x is the sample vector composed of the HOG features of the target, y is the regression target, and its shape is a two-dimensional Gaussian. is the Fourier transform of y.

[0053] For a Gaussian kernel, the element K in K xx ' indicates the kernel correlation between x and vector x' in X, which can be expressed by the following formula:

[0054]

[0055] X is the target sample obtained by cyclic shift, and the circulation of the vector can be obtained by the following permutation matrices P and Q.

[0056]

[0057] X=PxQ (5)

[0058] Circular shift actually performs various translations on samples within a certain frame to obtain a series of translated samples.

[0059] Getting trained to get After that, the tracker can track the target in the subsequent test frame. At this time, the response of the test sample It can be calculated by the following formula:

[0060]

[0061] In the formula K xz The Fourier transform of K xz is the core correlation of x and z, which can be expressed as follows:

[0062] K xz =φ(X)φ(z) (7)

[0063] K z is the kernel matrix composed of training samples and test samples. The elements in row i and column j are:

[0064]

[0065] By selecting the corresponding largest test sample, the position of the current target is obtained.

[0066] (IV) Using Kalman filter matching to correct KCF detection results

[0067] Kalman filtering is an algorithm that uses the state equation of a linear system and the system's observation data to optimally estimate the system state. Since the observation data is affected by noise and interference in the system, the system state estimation process can also be regarded as a filtering process.

[0068] The standard Kalman filter system equation is as follows:

[0069] w k =Aw k-1 +Bc k-1 +b k (9)

[0070] y k =Cw k +n k (10)

[0071] Formula (9) is the state equation, where w k is the state vector at the kth moment, c k-1 is the input at the kth moment, b k is the state transfer noise, A and B are system matrices, where A is the state transfer matrix. Formula (10) is the observation equation, C is the observation matrix, y k is the observation vector, n k is the observation error.

[0072] The Kalman filter is divided into two steps: prediction and update. The prediction formula is as follows:

[0073]

[0074] In the formula, w k -is the state prediction at time k, <w k-1 > is the Kalman filter estimate at time k-1, P k - is the Kalman increment prediction matrix, P k-1 is the estimation matrix of the Kalman filter at time k-1, and Q is the covariance matrix of the state transition noise.

[0075] The update formula of the Kalman filter is as follows:

[0076]

[0077] Where R is the covariance matrix of the observation error.

[0078] Through equations (11) and (12), the position obtained by KCF can be corrected to improve the tracking accuracy of the KCF algorithm.

[0079] In the process of tracker matching, the Kalman filter is used to predict the position of the target in the next frame according to the position of the current target. The tracker is updated by matching the distance between the predicted position and the detected position in the next frame.

[0080] (V) Target sperm determination

[0081] The above method is used to achieve real-time tracking of sperm. Based on the position information of each frame of the sperm tracker, the displacement of the sperm on the image during the adjacent frames is calculated to obtain the speed of sperm movement. The calculation method is as follows:

[0082] v k =x k -x k-1 (13)

[0083] Where x k is the sperm position, v k The speed of sperm.

[0084] The present invention calculates the average speed of sperms within 200 frames, and selects sperms with the largest average speed as target sperms for extraction.

[0085] (V) Sperm extraction based on deep reinforcement learning

[0086] Deep reinforcement learning framework design: The present invention uses a deep reinforcement learning method to realize the control of the extraction needle controller. Compared with the traditional control method, the method based on deep reinforcement learning has the advantages of optimal global strategy, good environmental adaptability, and high intelligence. Since the state information of the extraction needle controller is composed of the extraction needle and sperm position information, considering that the position information is continuous and the control amount of the stepper motor can also be regarded as continuous, the present invention uses a deep reinforcement algorithm in a continuous space-DDPG algorithm as a learning algorithm for target sperm extraction. The DDPG algorithm can realize the agent strategy learning in a continuous space and output a continuous control amount. The design of the DDPG algorithm includes the design of the extraction needle state, the design of the extraction needle behavior, the design of the reward function, and the design of the neural network. The neural network of DDPG includes two types of networks: actor network and critical network. The actor network can output the corresponding behavior according to the current state of the extraction needle. The critical network is used to evaluate the rationality of the behavior taken by the actor network during training, and its output is the evaluation value of the state-behavior. When the training is completed, the actor network can output the optimal strategy behavior a in the current state. In the training process, both the actor and critical networks include two networks, namely the policy network and the target network. The present invention uses c_actor_net to represent the policy actor network, t_actor_net to represent the target actor network, c_critical_net to represent the policy critical network, and t_critical_net to represent the target critical network.

[0087] For the actor network, the strategy optimization formula is as follows:

[0088]

[0089] In the formula, is the optimization target gradient of c_actor_net, To find the gradient operation, θ Q is the parameter of c_critical_net, θ μ is the parameter of c_actor_net, μ represents the c_actor_net network, Q represents the c_critical_net network, s k is the state at time k, a is the state s k The behavior performed below.

[0090] For critical networks, the learning objectives are as follows:

[0091] y=r k +γQ'(s k+1 ,μ'(sk+1 |θ μ' )|θ Q' ) (15)

[0092] Where y is the learning target, Q' represents t_critical_net, θ Q' represents the parameters of t_critical_net, μ' represents t_actor_net, θ μ' represents the parameters of t_actor_net, γ represents the discount factor of past experience, and s k+1 is the state at time k+1, r k For state s k Next, execute behavior a to transfer to state s k+1 The return received.

[0093] Based on formula (15), the learning error of the critical network is:

[0094] e=yQ(s k ,a) (16)

[0095] The learning of critical network and actor network can be completed through equations (14), (15) and (16).

[0096] State design: Extract the needle position G detected by the current frame image k 、sperm position x k and the sperm position x of the last test k-1 Construct observation state

[0097] s k =(G k ,x k-1 ,x k ) (14)

[0098] Where s k It changes with the movement of the target sperm and the movement of the retrieval needle.

[0099] Behavioral design: Considering that the extraction needle controller is controlled by three stepper motors, three control quantities u1, u2 and u3 are designed to control the stepping speed of the three motors respectively, and u i The value of is within the interval (-1, 1). Since the stepper motor controls the step angle by the number of pulses, the present invention obtains the control amount u i After that, you need to i Converted into the control pulse number of the stepper motor, the conversion method is as follows:

[0100]

[0101] Where npulse is the number of pulses, θ s is the step angle, which indicates the step angle of the stepper motor generated by one pulse. pulse When n is positive, the stepper motor rotates clockwise. pulse When it is negative, the stepper motor rotates counterclockwise.

[0102] The k-th control behavior of the extraction needle controller can be expressed as follows:

[0103] a=u=(u1,u2,u3) (16)

[0104] Reward function: The reward function in reinforcement learning is used to evaluate the current state s k The rationality of the action a taken by the controller. In addition to giving the reward for each control of the controller, the present invention also takes into account that the target sperm to be extracted is moving and the sperm may be blocked by the extraction needle, and further gives the reward under the condition of blockage. The entire reward function is designed as the following formula:

[0105]

[0106] In the formula, c pe is a constant, f e It is the extraction success flag. When the extraction is successful, e Set to true, otherwise set to false; d ε is the extractable range threshold; The distance between the extraction needle and the target sperm at the kth control is calculated by the following formula:

[0107]

[0108] Formula (16) gives the evaluation criteria of the behavior. When the target sperm is not detected, a reward of -1 is given. In the case where the target sperm can be detected, when the extraction needle is close to the target sperm, the present invention gives a reward according to the degree of proximity. When the extraction needle enters the extractable range, the extraction needle performs the extraction operation. If the target sperm is extracted, a reward of 1 is given to confirm the correctness of the entire control process. If it cannot be extracted, a reward of 0 is given, indicating that for some reason, it is difficult for the extraction needle to extract the target sperm in the current state. Neural network: Due to the state s k By G k ,x k-1 ,x k The network consists of three position data, and each position data is 2-dimensional. The behavior a consists of three one-dimensional control data u1, u2, and u3. Therefore, the input of the actor network is 6-dimensional and the output is 3-dimensional. The input of the critical network is 9-dimensional and the output is 1-dimensional.

[0109] For the actor network, the present invention designs a 3-layer BP neural network, the input layer has 6 neurons, the activation function is the ReLU function, the hidden layer has 100 neurons, the activation function is the ReLU function, the output layer has 3 neurons, and the activation function is the Tanh function.

[0110] For the critical network, the present invention also designs a 3-layer BP neural network, the input layer is 9 neurons, the activation function is the ReLU function, the hidden layer is 100 neurons, the activation function is the ReLU function, the output layer is 1 neuron, there is no activation function, and direct linear output.

[0111] Training method: The training method of deep reinforcement learning includes the following steps,

[0112] (1) Create two neural networks, actor and critical, calculate the initial observation state s0 according to formula (14), and use the created actor neural network to calculate the initial behavior a based on the current observation state;

[0113] (2) Based on formula (15), the control amount of behavior a is converted, and then the position of the extraction needle is adjusted;

[0114] (3) Calculate the latest observed state s of the target sperm k ;

[0115] (4) Calculate the reward value r of the last action according to formula (17): k ;

[0116] (5)Tuple (s k-1 ,s k ,a,r k ) is stored in a sample pool;

[0117] (6) Updates k-1 ,s k-1 =s k , and obtain the new behavior a through the neural network;

[0118] (7) determining whether the extraction needle has moved to the extraction range, and if so, executing the extraction operation;

[0119] (8) Randomly extract a fixed number of sample tuples from the sample pool for training the actor and critical networks;

[0120] (9) If the number of times the extraction needle is controlled exceeds the preset number, go to step (10), otherwise go to step (2);

[0121] (10) If the number of training rounds does not reach the preset number, go to step (1); otherwise, end the training and save the trained neural network model.

Claims

1. A sperm tracking and automatic extraction method based on machine learning, characterized in that: The following steps are involved: Step 1: Use an electron microscope to obtain real-time image data of the sperm sample and transmit the image to a computer at a transmission frequency of 20 FPS; Step 2: On the computing platform, the trained target detection network is used to detect the observed sperm image to obtain the location of the sperm; Step 3: Update the KCF tracker using the sperm position data obtained in step 2; Step 4: For the sperm that was missed in step 2, the estimated position of the missed sperm is obtained by using the KCF algorithm; Step 5: Use Kalman filter to correct the estimated position of sperm obtained in step 4; Step 6, using the corrected sperm position data obtained in step 5 to update the KCF tracker of the missed sperm; Step 7, calculating the displacement of the sperm on the image during adjacent frames based on the position information of each frame of the sperm tracker, thereby obtaining the speed of the sperm movement; Step 8: Calculate the average movement speed of sperm within 10 seconds of observation, select the sperm with the fastest speed as the target for extraction, and use x to track the position of the target sperm. k express; Step 9: Use the trained object detection network to detect the position of the sperm extraction needle in the current image and use G k Indicates that detection is performed every 4 frames; Step 10: Use the currently detected extraction needle position G k 、sperm position x k and the sperm position x of the last test k-1 Construct observation state s k =(G k ,x k-1 ,x k ); Step 11: Observe the state s k Input the trained deep reinforcement learning network model, and the neural network model outputs the control amount u = (u1, u2, u3) of the sperm extraction needle; Step 12: The control value u is transmitted to the extraction needle controller. The controller controls three stepping motors according to the three control values ​​u1, u2 and u3 respectively to adjust the extraction needle in the x, y and z directions. Step 13: If the extraction needle moves to the extractable range of the target sperm, the extraction needle is controlled to perform the extraction operation. If the target sperm is extracted, the control ends; if the extraction needle does not reach the extractable range of the target sperm or the extraction needle does not extract the target sperm after performing the extraction operation, jump to step 1.

2. The method for sperm tracking and automatic extraction based on machine learning according to claim 1, characterized in that: After obtaining the real-time image data of the sperm sample in step 1, the image sample set is classified. Some samples are used as training sets and validation sets to train the target detection network, and the target detection network adopts the YOLO-V4 network model; some samples are used as test sets, and the trained YOLO-V4 network model is used for testing.

3. The method for sperm tracking and automatic extraction based on machine learning according to claim 1, characterized in that: The training method of the deep reinforcement learning model in step 11 includes the following steps: Step 11.1, based on step 8, step 9, and step 10, calculate the initial observation state s0, and based on the initial observation state s0, use the created BP neural network to calculate the initial behavior a; Step 11.2, the extraction needle controller executes behavior a to complete the position adjustment of the extraction needle; Step 11.3: Calculate the latest observed state s of the target sperm k ; Step 11.4: Calculate the behavior reward value r based on the reward function k ; Step 11.5: Substitute the tuple (s k-1 ,s k ,a,r k ) is stored in a sample pool; Step 11.6, Update s k-1 ,s k-1 =s k , and calculate the behavior a of the new state through the BP neural network; Step 11.7, when the extraction needle moves to the extraction range, the extraction operation is performed; Step 11.8, randomly select a fixed-size sample from the sample pool for training the neural network; Step 11.9: If the control times of the extraction needle exceed the preset times, go to step 11.10; otherwise, go to step 11.2; Step 11.10: If the number of training rounds does not reach the preset number, go to step 11.1; otherwise, end the training and save the trained neural network model.

4. The training method of the deep reinforcement learning model according to claim 3, characterized in that: The return function in step 11.4 is: In the formula, c pe is a constant, f e It is the extraction success flag. When the extraction is successful, e Set to true, otherwise set to false; d ε is the extractable range threshold; The distance between the extraction needle and the target sperm at the kth control is calculated by the following formula: x k is the sperm position, G k , is the extraction needle position currently detected.

5. The method for sperm tracking and automatic extraction based on machine learning according to claim 1, characterized in that: In step 12, the three control quantities u1, u2 and u3 are represented by u i , and u i The value of is in the interval (-1, 1), and u i Converted into the control pulse number of the stepper motor, the conversion method is as follows: Where n pulse is the number of pulses, θ s is the step angle, which indicates the step angle of the stepper motor generated by one pulse. pulse When n is positive, the stepper motor rotates clockwise. pulse When it is negative, the stepper motor rotates counterclockwise.

Citation Information

Patent Citations

  • Sperm morphology detection method and device based on image technology

    CN111563550A

  • Multi-target sperm real-time monitoring method based on deep learning

    CN112150415A

  • Sperm quality detection method based on big data and image tracking

    CN112113962A

  • Sperm survival rate detection method and device

    CN112184708A

  • Automatic sperm morphology analysis method based on multi-scale feature fusion

    CN113313678A

Cited By

  • Insect sperm detection tracking method and system based on fine-grained subspace perception

    CN120599609A

  • An insect sperm detection and tracking method and system based on fine-grained subspace perception

    CN120599609B

  • Dynamic screening and trajectory tracking method and system for accurate evaluation of sperm motility

    CN121861451A

  • A dynamic screening and trajectory tracking method and system for precise assessment of sperm motility

    CN121861451B