Egg cytoplasm single sperm injection method based on image recognition

Abstract

The invention discloses an egg cytoplasm single sperm injection method based on image recognition, and belongs to the technical field of assisted reproduction, the method comprises the following steps: constructing a high-quality sperm evaluation feature tag, screening high-quality sperms by using a convolutional neural network model, and meanwhile, carrying out image acquisition and processing on ovum cells; obtaining an ovum edge image and an internal organelle segmentation image; in the aspect of injection point selection, through image fusion and analysis, multiple parameters such as cytoplasm gray scale, mitochondrial quantity and distance from cell nucleus are comprehensively considered, potential injection points are evaluated, evaluation values are calculated, and finally the optimal injection point is determined; according to the invention, the precision of sperm screening and ovum injection site selection is realized, the success rate of ovum cytoplasm single sperm injection is obviously improved, the problem of fertilization failure caused by poor sperm quality or inaccurate injection site is reduced, and a powerful technical support is provided for the development of an assisted reproduction technology.

Description

Technical Field

[0001] The present invention belongs to the technical field of assisted reproduction, and in particular relates to an intracytoplasmic sperm injection method based on image recognition. Background Art

[0002] During intracytoplasmic sperm injection (ICSI), sperm selection, egg fixation, and optimal injection site selection play a decisive role in fertilization success.

[0003] Traditional sperm screening methods rely heavily on experience, using simple indicators like sperm motility and morphology to determine sperm quality. This is highly subjective and has limited accuracy. This can lead to the misjudgment of a large number of potentially high-quality sperm, impacting fertilization success rates and preventing the full utilization of precious sperm resources.

[0004] Before the injection operation, how to accurately determine the optimal injection site of the egg has always been one of the difficulties of intracytoplasmic sperm injection technology. An inaccurate injection site may result in ineffective sperm fertilization and even damage the egg. Therefore, we propose an intracytoplasmic sperm injection method based on image recognition. Summary of the Invention

[0005] The purpose of the present invention is to provide an intracytoplasmic sperm injection method based on image recognition to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: an intracytoplasmic sperm injection method based on image recognition, comprising the following steps:

[0007] Step S1: Establishing high-quality sperm evaluation feature labels, building and training a convolutional neural network model, using the trained model to analyze sperm sample images, and screening sperm that meet high-quality standards for oocyte injection;

[0008] Step S2: collecting images of the egg cells to be injected with sperm, and filtering, edge extraction, and segmentation of the collected egg images to obtain egg edge images and internal organelle segmentation images;

[0009] Step S3: Perform image fusion processing on the egg edge image and the egg internal organelle segmentation image to obtain an egg fusion image, analyze the egg fusion image to obtain the injection area, set several potential injection points in the injection area, analyze the evaluation deviation value, injection nucleus distance deviation value and fit evaluation value corresponding to each potential injection point, obtain the evaluation value, and determine the optimal injection point for sperm injection based on it.

[0010] Preferably, the specific steps of establishing high-quality sperm evaluation feature labels and constructing and training the convolutional neural network model in step S1 are:

[0011] Step S101: Establish high-quality sperm evaluation feature tags; the evaluation feature tags include: Head shape: E1: Oval, E2: Symmetric circle; E3: Head length-width ratio; Tail shape: E4: Slender and straight, E5: Without curl, E6: Without entanglement; E7: Tail length.

[0012] Step S102: Establish a 6-layer convolutional neural network model, including an input layer, a first convolutional layer, a second convolutional layer, a third convolutional layer, a first fully connected layer, and a second fully connected layer;

[0013] The input layer includes a compressor and a preprocessor;

[0014] The compressor converts the image height and width to 1024×1024 pixels by compressing and cropping the original image;

[0015] The preprocessor of the first convolutional layer filters the input picture through compression cropping and Gaussian transformation. The Gaussian transformation formula is: where σ is the Gaussian standard deviation, * is the convolution operator, μ(x, y) is the Gaussian convolution kernel; f(x, y) is the input original image, and y0(x, y) is the output image of the input layer;

[0016] The first, second, and third convolutional layers all include a convolutional layer, an activation layer, and a pooling layer;

[0017] The first convolutional layer contains n1 nodes, and the output formula of the first convolutional layer is: where j1 = 1, 2, 3,..., n1; where * is the convolution operator; where is the weight factor of the first convolutional layer, is the bias factor of the first convolutional layer; where F is the activation function of the activation layer;

[0018] The second convolutional layer contains n2 nodes, and the output formula of the second convolutional layer is: where j2 = 1, 2, 3,..., n2; where is the weight factor of the second convolutional layer, is the bias factor of the second convolutional layer;

[0019] The third convolutional layer contains n3 nodes, and the output formula of the third convolutional layer is: where j3 = 1, 2, 3,..., n3; where is the weight factor of the third convolutional layer, is the bias factor of the second convolutional layer;

[0020] The pooling layers in the first, second, and third convolutional layers perform noise reduction and redundant data removal through average pooling operations. The so-called average pooling is to take the average value of the pixel point output results in the pooling area;

[0021] Preferably, the implementation processes of the first fully connected layer and the second fully connected layer are as follows:

[0022] The first fully connected layer contains n4 nodes, and its output formula is: where j4 = 1, 2, 3,..., n4; where is the weight factor of the first fully connected layer, is the bias factor of the first fully connected layer;

[0023] The second fully connected layer contains 7 nodes, and its output formula is where p = 01, 02, 03,..., 7; where W P is the weight factor of the second fully connected layer, is the bias factor of the second fully connected layer;

[0024] Step S103: Collect the photos related to the 7 sperm evaluation feature label elements established, with the number of photos corresponding to each feature label element being k, for a total of d photos; number each photo, and the numbering symbol is q, q = 1, 2, 3,..., d; d = 7k;

[0025] Perform the evaluation label marking work on each of the m photos one by one; for the photo containing a certain evaluation label element, set the evaluation label Ep corresponding to the photo to 1; otherwise, set Ep = 0; where p = 01, 02, 03,..., 7;

[0026] Step S104: Input the d photos marked with evaluation labels as the original images into the 6-layer convolutional neural network model, and make the output value of the 6-layer convolutional neural network model corresponding to each original image be specifically equal to the evaluation label Ep with the same numbering symbol p;

[0027] Perform the training operation on the convolutional neural network, and record all the weight factors and bias factors obtained after training at the end of training;

[0028] Test the convolutional neural network: Repeat step S103 to obtain m new photos as the original images and input them into the 6-layer convolutional neural network after training; then record the obtained output values Fp = F01, F02, F03,..., F7;

[0029] Through the formula Calculate the test success rate; if the test success rate S is greater than the preset threshold, it is determined that the training is successful, output all the weight factors and bias factors obtained after training and enter step S105;

[0030] Otherwise, it is determined that the training fails, and step S104 is re-run until the condition is met: the test success rate S is greater than the preset threshold.

[0031] Preferably, in step S1, the specific steps of using the trained model to analyze the sperm sample image and screening out the sperm that meets the high-quality standard for egg cell injection are as follows:

[0032] S105: Place the sperm sample in the culture medium, and use a high-resolution microscope and a supporting image acquisition device to collect images of the sperm sample placed in the culture medium, and collect a total of g sperm images;

[0033] Take the g collected sperm images one by one as the original images and input them into the 6-layer neural network. For each sperm image, a set of output values F01, F02, F03,..., F07 corresponding to 7 sperm evaluation feature labels are output; for each output value, compare it with the preset threshold. If Fp is greater than the preset threshold, it indicates that the sperm meets the evaluation of high-quality sperm in the characteristics described by the corresponding feature label. At this time, add a field named high-quality score to the metadata of the sperm image corresponding to the output value, with an initial value of 0. When the high-quality sperm evaluation condition is met, increase the value of this field by 1;

[0034] For each sperm image, count its corresponding high-quality score, and then add up all the high-quality scores of the g sperm images and calculate their average value to obtain the sperm judgment value. If the sperm judgment value is greater than or equal to the preset threshold, mark the sperm as high-quality sperm and use it as the sperm for egg cell injection; if the sperm judgment value is less than the preset threshold, it is determined as non-high-quality sperm and not used for egg cell injection. Then re-select the sperm sample, and substitute the image of the re-selected sperm sample into the neural network to evaluate again whether it is high-quality sperm, and so on until high-quality sperm is screened out.

[0035] Preferably, the specific steps of filtering and edge extraction of the collected egg image to obtain the egg edge image in step S2 are as follows:

[0036] Step S201: Collect images of the egg cells to be injected with sperm placed in the culture medium;

[0037] Step S202: Mark the collected egg image as I(x, y), and use the two-dimensional Gaussian function G(x, y, σ) as the convolutional layer for the egg image I(x, y). Its expression is: Let σ = 2; through the convolution operation: I G (x, y) = G(x, y, σ) * I(x, y) = ∑ m ∑ nG(x, y, σ)I(x - m, y - n) to obtain the filtered egg image, denoted as the egg filtered image I G (x, y); where m and n are offsets;

[0038] Step S203: Calculate the horizontal and vertical gradient components of each pixel in the egg filtered image I G (x, y), denoted as G x (x, y) and G y (x, y), using the formula: to obtain the gradient magnitude M(x, y) of the pixel, using the formula: to obtain the direction θ(x, y) of the gray - level gradient of each pixel;

[0039] For each pixel, along its corresponding gray - level gradient direction θ(x, y), find two adjacent pixels. Compare the gradient magnitude of the current pixel with the gradient magnitudes of these two adjacent pixels. If the gradient magnitude of the pixel is greater than or equal to the gradient magnitudes of the adjacent pixels in its gradient direction, retain the magnitude of the pixel; otherwise, set its magnitude to zero. By performing the above non - maximum suppression operation on each pixel in the egg image I G (x, y), the egg edge image E(x, y) is obtained.

[0040] Preferably, the specific steps for segmenting the collected egg image to obtain the internal organelle segmentation image in step S2 are:

[0041] Step S204: Use the Otsu algorithm to calculate the gray - level threshold of the egg filtered image I G (x, y) to obtain the gray - level segmentation value. For each pixel in the egg filtered image I G (x, y), compare its corresponding gray - level value with the gray - level segmentation value. If the gray - level value corresponding to the pixel is greater than the gray - level segmentation value, mark the pixel as a foreground pixel, represented by white; if the gray - level value corresponding to the pixel is less than or equal to the gray - level segmentation value, mark the pixel as a background pixel, represented by black. After marking all pixels, recombine the pixels with the marked foreground or background attributes to obtain the preliminary segmentation image S T (x, y);

[0042] S205: First perform an erosion operation on the preliminary segmentation image S T (x, y), and then perform a dilation operation to obtain the egg internal organelle segmentation image S(x, y).

[0043] Preferably, the specific steps for performing image fusion processing on the egg edge image and the egg internal organelle segmentation image to obtain the egg fusion image in step S3 are:

[0044] Step S301: For the edge image of the egg and the segmented image of the internal organelles of the egg, respectively, take the upper left vertex of each image as the coordinate origin, define the horizontal right direction as the positive direction of the x-axis, and the vertical downward direction as the positive direction of the y-axis, and construct a two-dimensional Cartesian coordinate system with pixels as the unit.

[0045] Use the image fusion technology to fuse the edge image of the egg and the segmented image of the internal organelles of the egg; during the fusion process, based on the established two-dimensional Cartesian coordinate system above, ensure that the corresponding pixel points in the two images are accurately aligned in spatial position to obtain the egg fusion image.

[0046] Preferably, the specific steps for analyzing the egg fusion image in step S3 to obtain the injection area are as follows:

[0047] Step S302: Use the image recognition technology to identify the nucleus area in the egg fusion image, and at the same time analyze the central coordinates of the nucleus area by the geometric center method. Subsequently, take the central coordinates of the nucleus area as the center of the circle and the preset shortest distance from the nucleus injection as the radius to draw a circle, and the obtained circular area is marked as the first area.

[0048] S303: In the egg fusion image, based on the existing edge contour of the egg, evenly select several reference points at a fixed interval along the egg edge. For each reference point, take this point as the starting point and measure the length of the preset shortest distance from the egg edge along the direction perpendicular to the egg edge tangent, mark the corresponding position point, and then connect all the marked position points in sequence through a smooth curve to form a closed curve surrounding the egg edge; mark the area enclosed by the closed curve as the second area, and mark the area between the second area and the first area as the injection area.

[0049] Preferably, the specific steps for analyzing the deviation evaluation value, nucleus injection distance deviation value, and contract evaluation value corresponding to each potential injection point in step S3 to obtain the excellent evaluation value, and then determining the best injection point and finally performing sperm injection are as follows:

[0050] Step S304: Manually select several potential injection points in the injection area. For each potential injection point, obtain the coordinates (x0, y0) of the potential injection point in the egg fusion image. Take the potential injection point as the center and set a small rectangular area with a side length of 3 pixels, denoted as the evaluation area. Use the formula: to obtain the average gray value of the cytoplasm in the evaluation area, denoted as the evaluation gray value HD, where I(x, y) is the gray value of the pixel point (i, j) in the evaluation area. Preset the standard evaluation gray value interval, subtract the midpoint value corresponding to the preset standard evaluation gray value interval from the evaluation gray value, and take the absolute value, denoted as the deviation evaluation value PZ.

[0051] Step S305: Using image recognition technology, identify the number of mitochondria in the evaluation area, count the number of mitochondria in the evaluation area, preset multiple intervals of mitochondrial numbers, and each interval of mitochondrial numbers corresponds to a fitness value;

[0052] By matching the number of mitochondria in the evaluation area with all the intervals of mitochondrial numbers, output the corresponding fitness value, denoted as the fitness evaluation value QZ;

[0053] By substituting the coordinates of the potential injection point and the central coordinates of the cell nucleus area into the Euclidean distance formula for calculation, obtain the nucleus injection distance value, preset the optimal nucleus injection distance value, and by taking the absolute value of the difference between the nucleus injection distance value and the preset optimal nucleus injection distance value, obtain the nucleus injection distance deviation value ZP;

[0054] For each potential injection point, after normalizing its corresponding evaluation deviation value PZ, nucleus injection distance deviation value ZP, and fitness evaluation value QZ, use the formula: PY = QZ×w1 + ZP×w2 - QZ×w3 to obtain the evaluation excellence value PY, where w1 and w2 are preset weight coefficients;

[0055] Step S306: Screen out the potential injection point corresponding to the maximum evaluation excellence value from all the potential injection points, and mark this potential injection point as the optimal injection point;

[0056] The injector performs sperm injection according to the coordinates corresponding to the optimal injection site and the high-quality sperm selected in step S1.

[0057] Compared with the prior art, the beneficial effects of the present invention are:

[0058] (1) For the method and system for intracytoplasmic sperm injection based on image recognition, by constructing high-quality sperm evaluation feature tags, designing a neural network and training, using the trained model to analyze the collected sperm images, outputting a sperm evaluation value, if the sperm evaluation value is greater than or equal to the preset threshold, then mark this sperm as high-quality sperm and use it as the injection sperm for the egg cell; otherwise, determine it as non-high-quality sperm, do not use it for egg cell injection, and re-select sperm samples, and so on until high-quality sperm are screened out. Through this method, the sperm samples can be evaluated according to the multi-dimensional high-quality sperm evaluation label standard, effectively screening out high-quality sperm for egg cell injection, improving the fertilization success rate, and reducing the problem of fertilization failure caused by poor sperm quality.

[0059] (2) The intracytoplasmic sperm injection method and system based on image recognition perform a series of processes such as filtering, edge extraction, and segmentation on the image of the oocyte to be injected, obtain the edge image of the oocyte and the segmented image of internal organelles, establish two-dimensional Cartesian coordinate systems for the edge image of the oocyte and the segmented image of internal organelles respectively, and then perform fusion processing, laying a solid foundation for determining the optimal injection site of the oocyte and effectively ensuring the accuracy of the injection site selection during the intracytoplasmic sperm injection process.

[0060] (3) The intracytoplasmic sperm injection method and system based on image recognition construct a fused egg image, analyze the fused egg image based on image recognition technology, divide the injection area, and comprehensively consider the cytoplasmic gray level, the number of mitochondria, and the distance from the cell nucleus to comprehensively evaluate potential injection points and calculate a merit value, and finally determine the optimal injection point; this process realizes the scientific and precise selection of the injection site, effectively avoids the problems of ineffective sperm fertilization or damage to the oocyte caused by inaccurate injection sites in traditional methods, significantly improves the success rate of intracytoplasmic sperm injection, and provides strong technical support for the development of assisted reproductive technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a flowchart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0063] Embodiment 1

[0064] Please refer to Figure 1 , the present invention provides an intracytoplasmic sperm injection method based on image recognition, including; general steps, selecting high-quality sperm, then fixing the oocyte, and analyzing the optimal injection site of the oocyte.

[0065] S1: Establish a feature label for evaluating high-quality sperm, construct and train a convolutional neural network model, and use the trained model to analyze the sperm sample image to screen out sperm that meet the high-quality standard for oocyte injection. The specific process is as follows:

[0066] S10 first: Establish a feature label for evaluating high-quality sperm;

[0067] The evaluation feature label includes:

[0068] Head shape: E1: Oval, E2: Symmetric circle;

[0069] E3: Head length-width ratio 1.7 - 1.8;

[0070] Tail shape: E4: Long and straight, E5: No curl, E6: No entanglement;

[0071] Tail length: E7: Length 45 - 55 microns;

[0072] S102: Establish a convolutional neural network;

[0073] Establish a 6-layer convolutional neural network model, including an input layer, a first convolutional layer, a second convolutional layer, a third convolutional layer, a first fully connected layer, and a second fully connected layer;

[0074] The input layer contains a compressor and a preprocessor;

[0075] The compressor converts the image height and width to 1024×1024 pixels by compressing and cropping the original image;

[0076] The preprocessor of the first convolutional layer filters the input picture through compression cropping and Gaussian transformation. The Gaussian transformation formula is: where σ is the Gaussian standard deviation, * is the convolution operator, μ(x, y) is the Gaussian convolution kernel; f(x, y) is the original input image, and y0(x, y) is the output image of the input layer;

[0077] The first, second, and third convolutional layers all contain a convolutional layer, an activation layer, and a pooling layer;

[0078] The first convolutional layer contains n1 nodes. The output formula of the first convolutional layer is: where j1 = 1, 2, 3,..., n1; where * is the convolution operator; where is the weight factor of the first convolutional layer, is the bias factor of the first convolutional layer; where F is the activation function of the activation layer;

[0079] The second convolutional layer contains n2 nodes. The output formula of the second convolutional layer is: where j2 = 1, 2, 3,..., n2; where * is the convolution operator; where is the weight factor of the second convolutional layer, is the bias factor of the second convolutional layer; where F is the activation function of the activation layer;

[0080] The third convolutional layer contains n3 nodes. The output formula of the third convolutional layer is: where j3 = 1, 2, 3,..., n3; where * is the convolution operator; where is the weight factor of the third convolutional layer, is the bias factor of the second convolutional layer; where F is the activation function of the activation layer;

[0081] The activation function F of the activation layer in the first, second, and third convolutional layers all adopts the LeakyReLU function, and its function expression is: where z is the input value of the LeakyReLU function;

[0082] The pooling layer in the first, second, and third convolutional layers performs noise reduction and redundant data removal through average pooling operation. The so-called average pooling is to take the average value of the pixel point output results in the pooling area;

[0083] The first fully connected layer contains n4 nodes, and its output formula is: where j4 = 1, 2, 3,..., n4; where * is the convolution operator; where is the weight factor of the first fully connected layer, is the bias factor of the first fully connected layer; where F is the activation function of the activation layer;

[0084] The second fully connected layer contains 7 nodes, and its output formula is where p = 01, 02, 03,..., 7; where * is the convolution operator; where W P is the weight factor of the second fully connected layer, is the bias factor of the second fully connected layer;

[0085] S103: Training image collection;

[0086] Collect photos related to the 7 sperm evaluation feature label elements established. The number of photos corresponding to each feature label element is k, and there are a total of d photos; number each photo, and the numbering symbol is q, q = 1, 2, 3,..., d; where d = 7k;

[0087] Carry out the evaluation label marking work on each of the d photos one by one; for the photo containing a certain evaluation label element, let the evaluation label Ep corresponding to the photo be 1; otherwise let Ep = 0; where p = 01, 02, 03,..., 7;

[0088] S104: Neural network training and machine learning;

[0089] Input the d photos marked with evaluation labels as the original images into the 6-layer convolutional neural network model, and let the output values of the 6-layer convolutional neural network model corresponding to each original image be specifically equal to the evaluation label Ep with the same numbering symbol p;

[0090] Perform training operations on the convolutional neural network, and record all the obtained weight factors and bias factors after the training ends;

[0091] Test the convolutional neural network: Repeat step S103 to obtain d new photos as the original images and input them into the 6-layer convolutional neural network after the training ends; Subsequently, record the obtained output values Fp = F01, F02, F03,..., F7;

[0092] Through the formula Calculate the test success rate; If the test success rate S is greater than the preset threshold, it is determined that the training is successful, output all the obtained weight factors and bias factors and enter step S105;

[0093] Otherwise, it is determined that the training fails, and step S104 is re-run until the condition is met: the test success rate S is greater than the preset threshold;

[0094] S105: Model application;

[0095] Place the sperm sample in the culture medium, and collect images of the sperm sample placed in the culture medium through a high-resolution microscope and the supporting image acquisition device, and a total of g sperm images are collected;

[0096] Input the g collected sperm images one by one as the original images into the 6-layer neural network. For each sperm image, output a set of output values F01, F02, F03,..., F07 corresponding to 7 sperm evaluation feature labels; For each output value, compare it with the preset threshold. If Fp is greater than the preset threshold, it indicates that the sperm meets the evaluation of high-quality sperm in the characteristics described by the corresponding feature label. At this time, add a field named high-quality score to the metadata of the sperm image corresponding to this output value, with an initial value of 0. When the high-quality sperm evaluation conditions are met, increase the value of this field by 1;

[0097] For each sperm image, count its corresponding high-quality score, and then add up all the high-quality scores of the g sperm images and calculate their average value to obtain the sperm judgment value. If the sperm judgment value is greater than or equal to the preset threshold, mark the sperm as high-quality sperm and use it as the sperm for egg injection; If the sperm judgment value is less than the preset threshold, it is determined as non-high-quality sperm and is not used for egg injection. Then re-select the sperm sample, and substitute the image of the re-selected sperm sample into the neural network to evaluate again whether it is high-quality sperm, and so on until high-quality sperm is screened out.

[0098] It should be noted that by constructing high-quality sperm evaluation feature tags, designing a neural network and training it, the trained model is used to analyze the collected sperm images and output sperm evaluation values. If the sperm evaluation value is greater than or equal to the preset threshold, the sperm is marked as high-quality sperm and used as the sperm for injecting into the egg cell; otherwise, it is determined as non-high-quality sperm and not used for egg cell injection, and a new sperm sample is selected, and so on until high-quality sperm is screened out. Through this method, the sperm samples can be evaluated according to the multi-dimensional high-quality sperm evaluation label standards, effectively screening out high-quality sperm for egg cell injection, improving the fertilization success rate, reducing the problem of fertilization failure caused by poor sperm quality, and facilitating the development of assisted reproductive technology.

[0099] S2: Collect images of the egg cells to be injected with sperm, and perform filtering, edge extraction, and segmentation on the collected egg images to obtain the egg edge image and the internal organelle segmentation image. The specific process is as follows:

[0100] S201: Collect images of the egg cells to be injected with sperm placed in the culture solution, and preprocess the collected egg images using an image denoising algorithm for denoising;

[0101] S202: Mark the collected egg image as I(x, y). For the egg image I(x, y), use the two-dimensional Gaussian function G(x, y, σ) as the convolutional layer, and its expression is: Let σ = 2; through convolutional operation: I G (x, y) = G(x, y, σ) * I(x, y) = ∑ m ∑ n G(x, y, σ)I(x - m, y - n), to obtain the filtered egg image, denoted as the egg filtered image I G (x, y); where m and n are offsets;

[0102] S203: Calculate the horizontal and vertical gradient components of each pixel point in the egg filtered image I G (x, y), denoted as G x (x, y) and G y (x, y), and use the formula: to obtain the gradient magnitude M(x, y) of the pixel point, and use the formula: to obtain the direction θ(x, y) of the gray gradient of each pixel point;

[0103] For each pixel, along its corresponding gray - level gradient direction θ(x, y), find two adjacent pixels. Compare the gradient magnitude of the current pixel with the gradient magnitudes of these two adjacent pixels. If the gradient magnitude of the pixel is greater than or equal to the gradient magnitudes of the adjacent pixels in its gradient direction, retain the magnitude of the pixel; otherwise, set its magnitude to zero. By performing the above non - maximum suppression operation on each pixel in the egg image I G (x, y), the egg edge image E(x, y) is obtained;

[0104] S204: Use the Otsu algorithm to calculate the gray - level threshold of the egg filter image I G (x, y) to obtain the gray - level segmentation value. For each pixel in the egg filter image I G (x, y), compare its corresponding gray - level value with the gray - level segmentation value. If the gray - level value corresponding to the pixel is greater than the gray - level segmentation value, mark the pixel as a foreground pixel and represent it in white; if the gray - level value corresponding to the pixel is less than or equal to the gray - level segmentation value, mark the pixel as a background pixel and represent it in black. After marking all pixels, recombine the pixels with the marked foreground or background attributes to obtain the preliminary segmentation image S T (x, y); The preliminary segmentation image is to segment the egg part and background pixels in the egg filter image I G (x, y), distinguish the pixels belonging to the egg from the background pixels, and then extract the pixels belonging to the egg to form a new image;

[0105] S205: First perform an erosion operation on the preliminary segmentation image S T (x, y) to remove small protrusions and isolated points in the foreground part of the image, and then perform a dilation operation to obtain the egg internal organelle segmentation image S(x, y); The purpose of the dilation operation is to restore the foreground area and connect the broken parts, and make the blurred organelle boundaries clearer, thereby highlighting the structure of the organelles and making the image gradually show the segmentation effect of the internal organelles of the egg.

[0106] It should be noted that by processing the collected egg images, that is, optimizing the egg images from multiple dimensions; the image denoising algorithm can effectively remove noise interference in the collected images and improve image quality; the two-dimensional Gaussian function filter further smoothes the image and reduces detail noise; the Sobel operator and non-maximum suppression operation accurately extract the edge of the egg, providing a key basis for determining the egg contour; the Otsu algorithm realizes the effective segmentation of the egg and the background, highlighting the main body of the egg; the morphological processing of corrosion and expansion focuses on presenting the internal organelle structure of the egg; through the mutual cooperation of the above operations, a solid foundation is laid for the subsequent precise determination of the optimal injection site of the egg, which effectively guarantees the accuracy of the injection site selection during the intracytoplasmic sperm injection, thereby improving the fertilization success rate.

[0107] S3: Perform image fusion processing on the egg edge image and the egg internal organelle segmentation image to obtain the egg fusion image. Analyze the egg fusion image to obtain the injection area. Set several potential injection points in the injection area. Analyze the evaluation deviation value, injection nucleus distance deviation value, and fit evaluation value corresponding to each potential injection point to obtain the evaluation value. Based on this, determine the optimal injection point and finally perform sperm injection. The specific process is as follows:

[0108] S301: For the egg edge image and the egg internal organelle segmentation image, use the upper left corner vertex of each image as the coordinate origin, define the horizontal right direction as the positive direction of the x-axis, and the vertical downward direction as the positive direction of the y-axis, and construct a two-dimensional Cartesian coordinate system with pixels as the unit;

[0109] Image fusion technology is used to fuse the egg edge image and the segmented image of the egg's internal organelles. During the fusion process, based on the established two-dimensional Cartesian coordinate system, the corresponding pixels in the two images are accurately aligned in space to obtain an egg fusion image. The egg fusion image is a comprehensive image that includes the complete egg outline and the clear segmentation of the internal organelle boundaries.

[0110] S302: Using image recognition technology, identifying the cell nucleus region in the fusion image, and analyzing the center coordinates of the cell nucleus region using a geometric center method. Then, a circle is drawn with the center coordinates of the cell nucleus region as the center and the shortest distance from the cell nucleus injection as the radius. The resulting circular region is marked as the first region;

[0111] S303: In the oocyte fusion image, based on the existing oocyte edge contour, a number of reference points are evenly selected along the oocyte edge at a fixed interval. For each reference point, starting from this point, along the direction perpendicular to the tangent of the oocyte edge, measure the length of the shortest distance from the oocyte edge inward by a preset distance, mark the corresponding position points, and then connect all the marked position points in sequence through a smooth curve to form a closed curve surrounding the oocyte edge; mark the area enclosed by the closed curve as the second area, and mark the area between the second area and the first area as the injection area;

[0112] S304: Manually select a number of potential injection points in the injection area. For each potential injection point, obtain the coordinates (x0, y0) of the potential injection point in the oocyte fusion image. With the potential injection point as the center, set a small rectangular area with a side length of 3 pixels, denoted as the evaluation area. Using the formula: Obtain the average gray value of the cytoplasm in the evaluation area, denoted as the evaluation gray value HD, where I(i, j) is the gray value of the pixel point (i, j) in the evaluation area. Preset a standard evaluation gray value interval, calculate the difference between the evaluation gray value and the midpoint value corresponding to the preset standard evaluation gray value interval, and take the absolute value, denoted as the evaluation deviation value PZ; the smaller the evaluation deviation value corresponding to the potential injection point, the closer the average gray value of the cytoplasm at this potential injection point is to the middle level of the standard evaluation gray value interval, which also means that this potential injection point is more in line with the preset ideal standard in terms of cytoplasmic gray value, and the suitability of this point as an injection point may be higher;

[0113] S305: Use image recognition technology to identify the number of mitochondria in the evaluation area and count the number of mitochondria in the evaluation area. Preset multiple mitochondria number intervals, and each mitochondria number interval corresponds to a fit value, where the fit value can be set artificially, that is, the closer to the desired optimal interval, the larger the fit value;

[0114] By matching the number of mitochondria in the evaluation area with all mitochondria number intervals, output the corresponding fit value, denoted as the fit evaluation value QZ;

[0115] By substituting the coordinates of the potential injection point and the center coordinates of the nucleus area into the Euclidean distance formula for calculation, obtain the injection-nucleus distance value. Preset the optimal injection-nucleus distance value, calculate the difference between the injection-nucleus distance value and the preset optimal injection-nucleus distance value and take the absolute value, to obtain the injection-nucleus distance deviation value ZP;

[0116] For each potential injection point, after normalizing its corresponding evaluation deviation value PZ, injection-nucleus distance deviation value ZP, and fit evaluation value QZ, use the formula: PY = QZ×w1 + ZP×w2 - QZ×w3 to obtain the evaluation optimization value PY, where w1 and w2 are preset weight coefficients;

[0117] S306: Screen out the potential injection point corresponding to the maximum evaluation value from all potential injection points, and mark this potential injection point as the optimal injection point;

[0118] Finally, the injector can perform sperm injection according to the coordinates corresponding to the optimal injection site and the high-quality sperm selected in step S1.

[0119] It should be noted that constructing a two-dimensional Cartesian coordinate system provides a unified spatial reference for image fusion, ensuring that the egg fusion image can completely and accurately present the egg structure; determining the first and second regions and the injection region narrows the range of potential injection points; the multi-parameter evaluation of potential injection points, including cytoplasmic gray level, mitochondrial number, and distance from the cell nucleus, comprehensively considers various key factors inside the egg; screening the optimal injection point by calculating the evaluation value makes the selection of the injection site more scientific and reasonable, can effectively improve the success rate of intracytoplasmic sperm injection, and promotes the precise development of assisted reproductive technology.

[0120] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intracytoplasmic sperm injection method based on image recognition, characterized in that: It includes the following steps: Step S1: Establish high-quality sperm evaluation feature tags, construct and train a convolutional neural network model, and use the trained model to analyze sperm sample images to screen out sperm that meet the high-quality standards for egg cell injection; Step S2: Collect images of the egg cells to be injected with sperm, and perform filtering, edge extraction, and segmentation on the collected egg images to obtain the egg edge image and the internal organelle segmentation image; Step S3: Perform image fusion processing on the egg edge image and the internal organelle segmentation image of the egg to obtain a fused egg image, analyze the fused egg image to obtain the injection area, set several potential injection points within the injection area, analyze the evaluation deviation value, injection nucleus distance deviation value, and contract evaluation value corresponding to each potential injection point to obtain an excellent evaluation value, and determine the best injection point for sperm injection based on this; 2. The intracytoplasmic sperm injection method based on image recognition according to claim 1, wherein: The specific steps for establishing high-quality sperm evaluation feature tags, constructing, and training a convolutional neural network model in Step S1 are as follows: Step S101: Establish high-quality sperm evaluation feature tags; The evaluation feature tags include: Head shape: E1: Oval, E2: Symmetric circle; E3: Head length-width ratio; Tail shape: E4: Slim and straight, E5: No curl, E6: No entanglement; E7: Tail length; Step S102: Establish a 6-layer convolutional neural network model, including an input layer, a first convolutional layer, a second convolutional layer, a third convolutional layer, a first fully connected layer, and a second fully connected layer; The input layer contains a compressor and a preprocessor; The compressor converts the image height and width to 1024×1024 pixels by compressing and cropping the original image; The preprocessor of the first convolutional layer filters the input image through compression cropping and Gaussian transformation. The Gaussian transformation formula is: where σ is the Gaussian standard deviation, * is the convolution operator, μ(x, y) is the Gaussian convolution kernel; f(x, y) is the original input image, and y0(x, y) is the output image of the input layer; The first, second, and third convolutional layers all include a convolutional layer, an activation layer, and a pooling layer; The first convolutional layer contains n1 nodes, and the output formula of the first convolutional layer is: where j1 = 1, 2, 3,..., n1; where * is the convolution operator; where is the weight factor of the first convolutional layer, is the bias factor of the first convolutional layer; where F is the activation function of the activation layer; The second convolutional layer contains n2 nodes, and the output formula of the second convolutional layer is: where j2 = 1, 2, 3,..., n2; where is the weight factor of the second convolutional layer, is the bias factor of the second convolutional layer; The third convolutional layer contains n3 nodes, and the output formula of the third convolutional layer is: where j3 = 1, 2, 3,..., n3; where is the weight factor of the third convolutional layer, is the bias factor of the second convolutional layer; The pooling layers in the first, second, and third convolutional layers perform noise reduction and redundant data removal through average pooling operations. The so-called average pooling is to take the average value of the pixel point output results in the pooling area.

3. The intracytoplasmic sperm injection method based on image recognition according to claim 2, characterized in that: The implementation process of the first fully connected layer and the second fully connected layer is as follows: The first fully-connected layer contains n4 nodes, and its output formula is: where j4 = 1, 2, 3,..., n4; Among them is the weight factor of the first fully-connected layer, is the bias factor of the first fully-connected layer; The second fully connected layer contains 7 nodes, and its output formula is where p = 01, 02, 03,..., 7; where w P is the weight factor of the second fully connected layer, is the bias factor of the second fully connected layer; Step S103: Collect photos related to the 7 sperm evaluation feature tag elements established. The number of photos corresponding to each feature tag element is k, with a total of d photos; number each photo, and the numbering symbol is q, q = 1, 2, 3,..., d; d = 7k; Perform evaluation label marking work on each of the m photos one by one; for a photo containing a certain evaluation label element, let the evaluation label Ep corresponding to the photo be 1; otherwise, let Ep = 0; Where p = 01, 02, 03,..., 7; Step S104: Input the d photos marked with evaluation tags as original images into the 6-layer convolutional neural network model, and make the specific value of the output value of the 6-layer convolutional neural network model corresponding to each original image equal to the evaluation tag Ep with the same serial number p; ​ Perform training operations on the convolutional neural network, and record all the weight factors and bias factors obtained after training at the end of training; Test the convolutional neural network: Repeat Step S103 to obtain m new photos as the original images and input them into the 6-layer convolutional neural network after training; then record the obtained output values Fp = F01, F02, F03,..., F7; Calculate the test success rate through the formula Calculate the test success rate; if the test success rate S is greater than the preset threshold, it is determined that the training is successful, output all the obtained weight factors and bias factors, and enter step S105; Otherwise, determine that the training fails, and re-run Step S104 until the condition is met: The test success rate S is greater than the preset threshold.

4. The intracytoplasmic sperm injection method based on image recognition according to claim 3, characterized in that: In step S1, the specific steps of using the trained model to analyze the sperm sample image and screening out the sperm that meet the high-quality standard for egg cell injection are as follows: S105: Place the sperm sample in the culture medium, and use a high-resolution microscope and supporting image acquisition equipment to collect images of the sperm sample placed in the culture medium, and a total of g sperm images are collected; Take the g collected sperm images one by one as the original images and input them into the 6-layer neural network. For each sperm image, a set of output values F01, F02, F03,..., F07 corresponding to 7 sperm evaluation feature labels are output; for each output value, compare it with the preset threshold. If Fp is greater than the preset threshold, it indicates that the sperm meets the evaluation of high-quality sperm in the characteristics described by the corresponding feature label. At this time, add a field named high-quality score to the metadata of the sperm image corresponding to this output value, with an initial value of 0. When the high-quality sperm evaluation condition is met, increase the value of this field by 1; For each sperm image, count its corresponding high-quality score, and then add up all the high-quality scores of the g sperm images and calculate their average value to obtain the sperm judgment value. If the sperm judgment value is greater than or equal to the preset threshold, mark the sperm as high-quality sperm and use it as the injection sperm for the egg cell; if the sperm judgment value is less than the preset threshold, it is determined as non-high-quality sperm and not used for egg cell injection. Then select a new sperm sample, substitute the image of the newly selected sperm sample into the neural network, and evaluate again whether it is high-quality sperm. Repeat this process until high-quality sperm are screened out.

5. The intracytoplasmic sperm injection method based on image recognition according to claim 4, characterized in that: In step S2, the specific steps of filtering and edge extraction of the collected egg image to obtain the egg edge image are as follows: Step S201: Collect images of the egg cells to be injected with sperm placed in the culture medium; Step S202: Mark the collected egg image as I(x, y). Use the two-dimensional Gaussian function G(x, y) as the convolutional layer for the egg image I(x, y). Its expression is: Let σ = 2; Through the convolution operation: I G (x, y) = G(x, y, σ) * I(x, y) = ∑ m ∑ n G(x, y, σ)I(x - m, y - n), to obtain the filtered egg image, denoted as the egg filtered image I G (x, y); where m and n are offsets; Step S203: Calculate the egg filter graph I through the Sobel operator G The gradient components in the horizontal and vertical directions of each pixel point in (x, y) are denoted as G x (x, y) and G y (x, y), using the formula: Obtain the gradient magnitude M(x, y) of the pixel point, using the formula: Obtain the direction θ(x, y) of the gray-scale gradient of each pixel point; For each pixel, along its corresponding gray-scale gradient direction θ(x, y), find two adjacent pixels to this pixel, compare the gradient magnitude of the current pixel with the gradient magnitudes of these two adjacent pixels. If the gradient magnitude of this pixel is greater than or equal to the gradient magnitudes of the adjacent pixels in its gradient direction, retain the magnitude of this pixel; otherwise, set its magnitude to zero. By performing the above non-maximum suppression operation on each pixel in the egg image I G (x, y), the egg edge image E(x, y) is obtained.

6. The intracytoplasmic sperm injection method based on image recognition according to claim 5, characterized in that: In step S2, the specific steps of segmenting the collected egg image to obtain the internal organelle segmentation image are as follows: Step S204: Calculate the gray threshold of the egg filter image I G at (x, y) using the Otsu algorithm to obtain the gray scale segmentation value. For each pixel point G in the egg filter image I(x, y), compare its corresponding gray value with the gray scale segmentation value. If the gray value corresponding to the pixel point is greater than the gray scale segmentation value, mark the pixel point as a foreground pixel and represent it in white; if the gray value corresponding to the pixel point is less than or equal to the gray scale segmentation value, mark the pixel point as a background pixel and represent it in black. After marking all pixel points, recombine the pixel points with the marked foreground or background attributes to obtain the preliminary segmentation image S T (x, y); S205: Perform erosion operation on the preliminary segmented image S T (x, y) first, and then perform a dilation operation to obtain the segmented image S(x, y) of the internal organelles of the egg.

7. The intracytoplasmic sperm injection method based on image recognition according to claim 6, wherein: In step S3, the specific steps of performing image fusion processing on the egg edge image and the internal organelle segmentation image of the egg to obtain the egg fusion image are as follows: Step S301: For the egg edge image and the internal organelle segmentation image of the egg, respectively use the upper left vertex of their respective images as the coordinate origin, define the horizontal right direction as the positive x-axis direction, and the vertical downward direction as the positive y-axis direction to construct a two-dimensional Cartesian coordinate system in pixels; Use image fusion technology to fuse the egg edge image and the internal organelle segmentation image of the egg; During the fusion process, based on the two-dimensional Cartesian coordinate system established above, ensure that the corresponding pixel points in the two images are accurately aligned in space to obtain the egg fusion image.

8. The intracytoplasmic sperm injection method based on image recognition according to claim 7, characterized in that: In step S3, the specific steps of analyzing the egg fusion image to obtain the injection area are as follows: Step S302: Use image recognition technology to identify the nucleus area in the egg fusion image, and at the same time analyze the center coordinates of the nucleus area by the geometric center method. Then, with the center coordinates of the nucleus area as the center and the preset shortest distance from the nucleus to the injection as the radius, draw a circle, and the obtained circular area is marked as the first area; S303: In the egg fusion image, based on the existing egg edge contour, select a number of reference points evenly at a fixed interval along the egg edge. For each reference point, starting from this point, measure the length of the shortest distance from the egg edge at a preset distance inward along the direction perpendicular to the tangent of the egg edge, mark the corresponding position points, and then connect all the marked position points in sequence with a smooth curve to form a closed curve around the egg edge; mark the area enclosed by the closed curve as the second area, and mark the area between the second area and the first area as the injection area.

9. The intracytoplasmic sperm injection method based on image recognition according to claim 8, wherein: In step S3, the specific steps for analyzing the deviation evaluation value, nucleus injection distance deviation value, and fitting evaluation value corresponding to each potential injection point to obtain the excellent evaluation value and thereby determine the best injection point and finally perform sperm injection are as follows: Step S304: Manually select a number of potential injection points in the injection area. For each potential injection point, obtain the coordinates (x0, y0) of the potential injection point in the oocyte fusion image. With the potential injection point as the center, set a small rectangular area with a side length of 3 pixels, denoted as the evaluation area. Using the formula: Obtain the average gray value of the cytoplasm in the evaluation area, denoted as the evaluation gray value HD. Here, I(i, j) is the gray value of the pixel point (i, j) in the evaluation area. Preset a standard evaluation gray value interval. Subtract the midpoint value corresponding to the preset standard evaluation gray value interval from the evaluation gray value and take the absolute value, denoted as the evaluation deviation value PZ; Step S305: Use image recognition technology to identify the number of mitochondria in the evaluation area and count the number of mitochondria in the evaluation area. Preset multiple mitochondria number intervals, and each mitochondria number interval corresponds to a fitting value; By matching the number of mitochondria in the evaluation area with all mitochondria number intervals, output the corresponding fitting value, denoted as the fitting evaluation value QZ; By substituting the coordinates of the potential injection point and the center coordinates of the nucleus area into the Euclidean distance formula for calculation, obtain the nucleus injection distance value. Preset the optimal nucleus injection distance value, and by taking the absolute value of the difference between the nucleus injection distance value and the preset optimal nucleus injection distance value, obtain the nucleus injection distance deviation value ZP; For each potential injection point, after normalizing its corresponding deviation evaluation value PZ, nucleus injection distance deviation value ZP, and fitting evaluation value QZ, use the formula: PY = QZ×w1 + ZP×w2 - QZ×w3 to obtain the excellent evaluation value PY, where w1 and w2 are preset weight coefficients; Step S306: Screen out the potential injection point corresponding to the maximum excellent evaluation value from all potential injection points and mark this potential injection point as the best injection point; The injector performs sperm injection according to the coordinates corresponding to the best injection site and the high-quality sperm selected in step S1.

Images