A smart system for assisting embryo grading and its storage medium

Abstract

This invention provides an intelligent system and storage medium for assisting embryo grading. It acquires images of livestock embryos at at least one stage, from activation to pre-implantation, using fused cells. The image set is pre-processed and labeled. A sub-convolutional neural network is set up for each stage, and the sub-convolutional neural network is trained using the training set for each stage to obtain a trained sub-convolutional neural network for each stage. A convolutional layer is connected to the last convolutional layer of each trained sub-convolutional neural network, and the convolutional layer is then processed. i The output is flattened and then used for feature fusion, serving as the input to the fully connected layer. The structure and parameters of the last convolutional layer in each sub-convolutional neural network remain unchanged. Based on the training set, Conv... i The fully connected layers are retrained to obtain a trained neural network; the trained neural network is then used to grade embryos. This invention improves the intelligence of embryo grading and reduces the workload of technicians.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more particularly to an intelligent system for assisting embryo grading and its storage medium. Background Technology

[0002] Cloning is the process of creating an individual identical to the original through asexual reproduction using biotechnology. It was first proposed by a German scientist in 1938, and in 1996, British scientist Ian Wilmut cloned the world's first live-born sheep. Cloning technology has broad application prospects in endangered species conservation, biopharmaceuticals, tissue repair, and breeding.

[0003] In the breeding of livestock such as cattle, embryos are obtained by cloning somatic cells from donor cattle. These embryos are then transferred to recipient cattle to breed superior breeds. Compared to sexual reproduction, cloning technology significantly reduces breeding time and ensures the stability of superior breeds. To improve the success rate of cloning, before embryo transfer, operators classify the embryos under a microscope into grades such as A, B, and C. Based on the classification, the embryos are placed in different syringes, and high-quality embryos are prioritized for transfer. However, this grading process is mostly done manually. On the one hand, it is labor-intensive and inefficient; on the other hand, the grading standards are difficult to control and rely heavily on the operator's experience. How to quickly grade embryos is a pressing problem to be solved in the breeding of superior cattle. Summary of the Invention

[0004] To address the problems of lack of intelligent technology and high subjectivity in embryo grading, this invention provides an intelligent system to assist in embryo grading, comprising the following modules:

[0005] The sample set construction module is used to acquire embryo images of livestock fusion cells from activation to at least one stage before transplantation. Each stage includes at least one embryo image. The module constructs an image set corresponding to each stage, puts the preprocessed embryo images into the corresponding image set, and annotates the image set to obtain the training set, validation set, and test set for each stage.

[0006] The model building and training module is used to set up a sub-convolutional neural network for each stage, train the sub-convolutional neural network using the training set of each stage, and adjust the hyperparameters using the validation set corresponding to each stage to obtain the trained sub-convolutional neural network for each stage. A convolutional layer Conv is then connected to the last convolutional layer of each trained sub-convolutional neural network. i For each convolutional layer Conv iThe output is flattened and then subjected to feature fusion, serving as the input to the fully connected layer FC1. The structure and parameters of the last convolutional layer in each sub-convolutional neural network remain unchanged. Based on the training set, Conv... i Retrain the fully connected layer to obtain a trained neural network;

[0007] The livestock embryo grading module is used to input images of embryos with fused cells to be identified into the trained neural network to obtain embryo grading information.

[0008] Preferably, the convolutional layer Conv i The number and size of the convolutional kernels are determined based on the stage and the number of nodes in the fully connected layer FC1, specifically as follows:

[0009] Obtain the stage number K and the number of nodes M in the fully connected layer FC1, and then determine the number of nodes allocated to the fully connected layer FC1 for each stage. α is the weight for assigning the number of FC1 nodes in the fully connected layer to each stage;

[0010] Choose one or more from 1×1, 3×3, and 5×5 as the convolutional layer Conv. i The middle convolution kernel makes the convolutional layer Conv i The number of feature points in the output feature map and the number of nodes w i same.

[0011] Preferably, the step of adjusting Conv based on the training set i Retraining with the fully connected layer yields a trained neural network, specifically:

[0012] Each time, training samples from the training set of each stage are selected and input into the sub-neural network corresponding to that stage, and Conv... i The first stage of training is performed with the fully connected layer. If the accuracy is greater than the first threshold, the second stage of training is started. Otherwise, the number of nodes in the fully connected layer FC1 is increased, and a fully connected layer is added to the fully connected layer FC1 until the accuracy is greater than the first threshold.

[0013] The second stage of training involves using training sets from different stages of the same embryo for each Conv training session. i Training is performed with fully connected layers. When the accuracy exceeds the first threshold, the second stage of training is stopped. Otherwise, the training sets of the same embryo at each stage are enhanced in the same way, and then the enhanced training sets are used for training again.

[0014] Preferably, the system further includes a calibration module;

[0015] The correction module is used to receive the graded probability values ​​output by each sub-convolutional neural network, calculate the standard deviation of all probability values, and when the standard deviation is greater than a second threshold, while displaying the graded information of the embryo, it also displays the image corresponding to the stage with the largest deviation from the average value of the probability values.

[0016] Preferably, setting up a sub-convolutional neural network for each stage specifically involves:

[0017] The visualization interface displays configuration items for configuring the sub-convolutional neural network. These configuration items include the stage name and the specific structure of the sub-neural network. The specific structure includes the order and number of convolutional layers, pooling layers, fully connected layers, and activation layers. The specific structure also includes the size and number of convolutional kernels in the convolutional layers, the number of fully connected layers, and the number of nodes.

[0018] Secondly, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the following method:

[0019] Step 1: Obtain embryo images of livestock fusion cells at at least one stage from activation to transplantation. Each stage includes at least one embryo image. Construct an image set corresponding to each stage, put the preprocessed embryo images into the corresponding image set, and label the image set to obtain the training set, validation set, and test set for each stage.

[0020] Step 2: Set up a sub-convolutional neural network for each stage, train the sub-convolutional neural network using the training set of each stage, and adjust the hyperparameters using the validation set corresponding to each stage to obtain the trained sub-convolutional neural network for each stage; connect a convolutional layer Conv to the last convolutional layer of each trained sub-convolutional neural network. i For each convolutional layer Conv i The output is flattened and then subjected to feature fusion, serving as the input to the fully connected layer FC1. The structure and parameters of the last convolutional layer in each sub-convolutional neural network remain unchanged. Based on the training set, Conv... i Retrain the fully connected layer to obtain a trained neural network;

[0021] Step 3: Input the embryo image of the fused cells to be identified into the trained neural network to obtain the embryo grading information.

[0022] Preferably, the convolutional layer Conv i The number and size of the convolutional kernels are determined based on the stage and the number of nodes in the fully connected layer FC1, specifically as follows:

[0023] Obtain the stage number K and the number of nodes M in the fully connected layer FC1, and then determine the number of nodes allocated to the fully connected layer FC1 for each stage. α is the weight for assigning the number of FC1 nodes in the fully connected layer to each stage;

[0024] Choose one or more from 1×1, 3×3, and 5×5 as the convolutional layer Conv. i The middle convolution kernel makes the convolutional layer Conv i The number of feature points in the output feature map and the number of nodes w i same.

[0025] Preferably, the step of adjusting Conv based on the training set i Retraining with the fully connected layer yields a trained neural network, specifically:

[0026] Each time, training samples from the training set of each stage are selected and input into the sub-neural network corresponding to that stage, and Conv... i The first stage of training is performed with the fully connected layer. If the accuracy is greater than the first threshold, the second stage of training is started. Otherwise, the number of nodes in the fully connected layer FC1 is increased, and a fully connected layer is added to the fully connected layer FC1 until the accuracy is greater than the first threshold.

[0027] The second stage of training involves using training sets from different stages of the same embryo for each Conv training session. i Training is performed with fully connected layers. When the accuracy exceeds the first threshold, the second stage of training is stopped. Otherwise, the training sets of the same embryo at each stage are enhanced in the same way, and then the enhanced training sets are used for training again.

[0028] Preferably, step four is included after step three;

[0029] Step four: Receive the graded probability values ​​output by each sub-convolutional neural network, calculate the standard deviation of all probability values, and when the standard deviation is greater than the second threshold, display the graded information of the embryo while simultaneously displaying the image corresponding to the stage with the largest deviation from the average value of the probability values.

[0030] Preferably, setting up a sub-convolutional neural network for each stage specifically involves:

[0031] The visualization interface displays configuration items for configuring the sub-convolutional neural network. These configuration items include the stage name and the specific structure of the sub-neural network. The specific structure includes the order and number of convolutional layers, pooling layers, fully connected layers, and activation layers. The specific structure also includes the size and number of convolutional kernels in the convolutional layers, the number of fully connected layers, and the number of nodes.

[0032] Finally, the present invention also provides an intelligent method for assisting embryo grading, the method comprising the following steps:

[0033] Step 1: Obtain embryo images of livestock fusion cells at at least one stage from activation to transplantation. Each stage includes at least one embryo image. Construct an image set corresponding to each stage, put the preprocessed embryo images into the corresponding image set, and label the image set to obtain the training set, validation set, and test set for each stage.

[0034] Step 2: Set up a sub-convolutional neural network for each stage, train the sub-convolutional neural network using the training set of each stage, and adjust the hyperparameters using the validation set corresponding to each stage to obtain the trained sub-convolutional neural network for each stage; connect a convolutional layer Conv to the last convolutional layer of each trained sub-convolutional neural network. i For each convolutional layer Conv i The output is flattened and then subjected to feature fusion, serving as the input to the fully connected layer FC1. The structure and parameters of the last convolutional layer in each sub-convolutional neural network remain unchanged. Based on the training set, Conv... i Retrain the fully connected layer to obtain a trained neural network;

[0035] Step 3: Input the embryo image of the fused cells to be identified into the trained neural network to obtain the embryo grading information.

[0036] In livestock cloning, fused cells continuously divide and grow from activation to embryo transfer, and this process affects the final cloning result. In the past, during embryo transfer, technicians could only observe the development of the embryo before transfer and could not capture the entire process. This invention acquires images of each stage of embryonic development and then uses a neural network to identify them, obtaining the final grading result. This reduces the workload of technicians in selecting embryos during embryo transfer, lightens their workload, and overcomes the subjective experience of different technicians. In addition, the visualized neural network configuration greatly improves the convenience of neural network configuration. Attached Figure Description

[0037] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0038] Figure 1A comparison image of high-quality and low-quality embryos;

[0039] Figure 2 This is a diagram of the convolutional neural network structure corresponding to one stage.

[0040] Figure 3 This is a diagram of a deep neural network structure.

[0041] Figure 4 This is a diagram of a deep neural network structure with hidden layers.

[0042] Figure 5 This is a flowchart of Example 2;

[0043] Figure 6 This is a flowchart of Example 3. Detailed Implementation

[0044] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Example 1

[0047] The quality of embryos directly affects the success or failure of cloning. Embryo morphology is the most important method for selecting embryos in cloning and in vitro fertilization (IVF), that is, selecting embryos based on their shape and appearance during development. This is also the most widely used embryo evaluation method internationally. At different stages of embryonic development, their morphology and cell number vary, and the standards for grading embryos also differ. Table 1 shows the grading standards for cleavage-stage embryos, where grade A is the best and grade D is the worst. Grades A and B can be used for transfer, while grade D is considered a discarded embryo.

[0048] Table 1

[0049]

[0050] Morphological characteristics of fused cells during their development from activation to pre-implantation are crucial for selecting embryos for transfer. In existing technologies, technicians often select embryos based on their pre-implantation morphological characteristics. This process only utilizes these morphological features and fails to incorporate information about the entire developmental process from activation to pre-implantation. To reduce the workload of technicians in selecting embryos and to improve the accuracy of embryo classification, this invention provides an intelligent system to assist in embryo grading. This system includes the following modules:

[0051] The sample set construction module is used to acquire embryo images of livestock fusion cells from activation to at least one stage before transplantation. Each stage includes at least one embryo image. The module constructs an image set corresponding to each stage, puts the preprocessed embryo images into the corresponding image set, and annotates the image set to obtain the training set, validation set, and test set for each stage.

[0052] In the cloning of livestock such as cattle, the nucleus of a somatic cell from the native animal is extracted and fused into an egg cell. After activation, and after a period of culture, usually several days, it is transplanted into a surrogate animal to complete the cloning process. From activation to transplantation, the fused cell, or recombinant cell, goes through several stages. Based on morphological characteristics, at a fine-grained scale, the pre-transplantation embryonic development process is divided into the pronuclear stage (also known as the zygote or single-cell embryo), two-cell stage, four-cell stage, eight-cell stage, morula stage, and blastocyst stage; at a broader scale, it is divided into the cleavage stage, morula stage, and blastocyst stage. Since embryonic development is time-dependent, development that is too fast or too slow can also affect embryo transplantation. In another specific embodiment, the stage division is determined by time, for example, D1-D3 is the first stage. This invention does not specifically limit the stage division. If a broad-granularity stage division meets the embryo grading requirements, then a broad-granularity stage division method can be used. If the embryo grading requirements are higher, or if, after identifying a sufficient number of high-quality embryos using a broad-granularity stage division, further selection from the best is desired, a fine-granularity stage division method can be used. A convolutional neural network can be trained to identify each stage.

[0053] For acquiring embryo images, time-lapse photography, also known as time-lapse imaging, can record the process of fused cells developing into a transplantable embryo, for example, by taking pictures of the embryo every 20 or 30 minutes.

[0054] Each stage will have at least one image, which forms a correspondence between embryo, stage, and image set. For example, A ij =[Aij [1], A ij [2], …, A ij [S] represents the picture set of the j-th stage of the i-th fusion cell, with a total of S pictures, 0 < j ≤ K, where i is the number of embryos in the sample, and K is the number of stages. Suppose there are 50,000 embryos with pictures at each stage. Then the pictures of the first stage of the 50,000 embryos form the sample set of the first stage, the pictures of the second stage form the sample set of the second stage, and so on until the sample set of the last stage is obtained. The sample set of each stage is further divided into a training set, a validation set, and a test set according to a certain proportion. The training set is used to train the neural network to determine relevant parameters, the validation set is used to adjust the hyperparameters of the neural network, and the test set is used to evaluate the model finally.

[0055] The model construction and training module is used to set a sub-convolutional neural network for each stage. As Figure 2 shown, the sub-convolutional neural network is trained using the training set of each stage, and the hyperparameters are adjusted through the validation set corresponding to the stage, so as to obtain the trained sub-convolutional neural network corresponding to each stage; a convolutional layer Conv i is connected to the last convolutional layer of each trained sub-convolutional neural network. After flattening the output of each convolutional layer Conv i , a feature fusion operation is performed, which is used as the input of the fully connected layer FC1, as Figure 3 shown; keeping the structure and parameters of the last convolutional layer and the last convolutional layer of each sub-convolutional neural network unchanged, Conv i and the fully connected layer are retrained according to the training set to obtain the trained neural network;

[0056] Convolution is used to extract the features of pictures. Appropriate convolutional kernels, the number of convolutional layers, the number of pooling layers, and their connection relationships are the key to picture feature extraction. In the present invention, different sub-convolutional neural network structures (using a visual configuration method) are used for embryo pictures at each stage, and then trained to obtain the most suitable convolutional process for this stage. Then, the output of the last convolutional layer of the sub-convolutional neural network most suitable for each stage is used as the input of the new convolutional layer Conv i corresponding to this sub-convolutional neural network. The new convolutional layer Conv i outputs a feature map, and the feature map is flattened and used as the input of the fully connected layer FC1. In a specific embodiment, the feature fusion operation performed after flattening the output of each convolutional layer Conv i specifically means first performing a Flatten operation on the output of each convolutional layer Conv i , and then performing a concat operation. In another specific embodiment, there is a hidden layer after the fully connected layer FC1, as Figure 4 As shown.

[0057] Then the training sets from each stage are input into the corresponding sub-convolutional neural networks to process Conv. i The fully connected layer is then retrained to obtain a trained neural network, where i represents the stage number. The first stage of the sub-convolutional neural network corresponds to Conv1, and so on.

[0058] The livestock embryo grading module is used to input images of embryos with fused cells to be identified into the trained neural network to obtain embryo grading information.

[0059] The evaluation criteria and morphological characteristics of embryos differ at different stages, and morphological characteristics become increasingly important as embryos progress, playing a greater role in the final classification. To reflect the contribution of different stages to embryo grading, in one specific embodiment, the convolutional layer Conv... i The number and size of the convolutional kernels are determined based on the stage and the number of nodes in the fully connected layer FC1, specifically as follows:

[0060] Obtain the stage number K and the number of nodes M in the fully connected layer FC1, and then determine the number of nodes allocated to the fully connected layer FC1 for each stage. α is the weight for assigning the number of FC1 nodes in the fully connected layer to each stage;

[0061] Assuming there are four phases, the fully connected layer FC1 has 300 nodes with a weight of 2, i.e., K=4, M=300, α=2. Then, according to the above formula, the number of fully connected layer FC1 nodes allocated to the first phase is w1=20, and similarly, w2=40, w3=80, w4=160.

[0062] Choose one or more from 1×1, 3×3, and 5×5 as the convolutional layer Conv. i The middle convolution kernel makes the convolutional layer Conv i The number of feature points in the output feature map and the number of nodes w i same.

[0063] Each sub-convolutional neural network corresponds to a convolutional layer Conv i By applying the Conv layer i Adjusting the kernel size and padding in the convolutional layers makes the Conv... i After convolution, the output feature map has the same number of nodes as the stage corresponding to the sub-convolutional neural network. In another embodiment, if after a preset number of adjustments, the convolutional layer Conv... i The number of feature points in the output feature map and the number of nodes w iIf they are the same, then the convolutional layer Conv with the number of feature points in the feature map being greater than the number of nodes and closest to the number of nodes is selected. i As the final convolutional layer, the first and last n feature points of the flattened feature map are discarded, ensuring that the number of remaining feature points is the same as the number of nodes.

[0064] To improve the training effect, the Conv is adjusted according to the training set. i Retraining with the fully connected layer yields a trained neural network. Specifically, each time, training samples from the training set for each stage are selected and input into the sub-neural network corresponding to that stage, and Conv... i The first stage of training is performed with the fully connected layer. If the accuracy is greater than the first threshold, the second stage of training is started. Otherwise, the number of nodes in the fully connected layer FC1 is increased, and a fully connected layer is added to the fully connected layer FC1 until the accuracy is greater than the first threshold.

[0065] The second stage of training involves using training sets from different stages of the same embryo for each Conv training session. i Training is performed with fully connected layers. When the accuracy exceeds the first threshold, the second stage of training is stopped. Otherwise, the training sets of the same embryo at each stage are enhanced in the same way, and then the enhanced training sets are used for training again.

[0066] After training each sub-convolutional neural network, the entire convolutional neural network needs to be trained. This training is divided into two stages. First, regardless of whether each stage originates from the same embryo, images from the same stage of all embryos are grouped together for training, forming B1, B2, ..., B... K In one training iteration, several images are selected from B1 and fed into the first sub-convolutional neural network, several images are selected from B2 and fed into the second sub-convolutional neural network, and so on; then a second training iteration is performed. Where B... i A collection of images representing the same stage of all embryos;

[0067] The second stage is to differentiate between different embryos, that is, in one training session, A... 11 A 12 A 1K The images in the image are fed into the corresponding sub-convolutional neural network, and then A... 21 A 22 A 2KThe images in Aij are fed into the corresponding sub-convolutional neural networks, and so on. It is important to note that not all images in Aij are input into the corresponding sub-convolutional neural networks. Instead, images are selected from Aij based on the input requirements of the sub-convolutional neural networks. If there are 50,000 images of embryos, and 40,000 images of embryos are selected for training, then the second stage requires 40,000 training iterations.

[0068] In practical applications, sometimes even though the embryo is classified as Category A, the pregnancy rate is still not good after the transfer. This may be because there is a problem with embryo development at a certain stage from activation to transfer. In order to facilitate technicians to obtain abnormal information and to facilitate the selection of the best embryos, the system also includes a correction module.

[0069] The correction module is used to receive the graded probability values ​​output by each sub-convolutional neural network, calculate the standard deviation of all probability values, and when the standard deviation is greater than a second threshold, while displaying the graded information of the embryo, it also displays the image corresponding to the stage with the largest deviation from the average value of the probability values.

[0070] One embryo is ultimately classified as category A, but one stage is classified as category C or even D, while the other stages are also category A; another embryo is also ultimately classified as category A, and every stage is classified as category A. Clearly, the second embryo is preferred for transfer. If a second embryo is not available, technicians need to examine the images corresponding to the stages classified as category C or even D and assess the severity to see if it will affect the transfer.

[0071] To facilitate the configuration of different sub-convolutional neural networks, in one specific embodiment, setting up a sub-convolutional neural network for each stage specifically involves:

[0072] The visualization interface displays configuration items for configuring the sub-convolutional neural network. These configuration items include the stage name and the specific structure of the sub-neural network. The specific structure includes the order and number of convolutional layers, pooling layers, fully connected layers, and activation layers. The specific structure also includes the size and number of convolutional kernels in the convolutional layers, the number of fully connected layers, and the number of nodes.

[0073] Users can configure sub-convolutional neural networks with a simple click. Furthermore, in another embodiment, the convolutional layers (Conv) corresponding to each sub-convolutional neural network can also be configured. i This ensures that the final output meets the node count requirement for the fully connected layer FC1 allocated to this sub-convolutional neural network.

[0074] Example 2

[0075] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the following methods, such as... Figure 5 As shown:

[0076] Step 1: Obtain embryo images of livestock fusion cells at at least one stage from activation to transplantation. Each stage includes at least one embryo image. Construct an image set corresponding to each stage, put the preprocessed embryo images into the corresponding image set, and label the image set to obtain the training set, validation set, and test set for each stage.

[0077] Step 2: Set up a sub-convolutional neural network for each stage, train the sub-convolutional neural network using the training set of each stage, and adjust the hyperparameters using the validation set corresponding to each stage to obtain the trained sub-convolutional neural network for each stage; connect a convolutional layer Conv to the last convolutional layer of each trained sub-convolutional neural network. i For each convolutional layer Conv i The output is flattened and then subjected to feature fusion, serving as the input to the fully connected layer FC1. The structure and parameters of the last convolutional layer in each sub-convolutional neural network remain unchanged. Based on the training set, Conv... i Retrain the fully connected layer to obtain a trained neural network;

[0078] Step 3: Input the embryo image of the fused cells to be identified into the trained neural network to obtain the embryo grading information.

[0079] Preferably, the convolutional layer Conv i The number and size of the convolutional kernels are determined based on the stage and the number of nodes in the fully connected layer FC1, specifically as follows:

[0080] Obtain the stage number K and the number of nodes M in the fully connected layer FC1, and then determine the number of nodes allocated to the fully connected layer FC1 for each stage. α is the weight for assigning the number of FC1 nodes in the fully connected layer to each stage;

[0081] Choose one or more from 1×1, 3×3, and 5×5 as the convolutional layer Conv. i The middle convolution kernel makes the convolutional layer Conv i The number of feature points in the output feature map and the number of nodes w i same.

[0082] Preferably, the step of adjusting Conv based on the training set i Retraining with the fully connected layer yields a trained neural network, specifically:

[0083] Each time, training samples from the training set of each stage are selected and input into the sub-neural network corresponding to that stage, and Conv... i The first stage of training is performed with the fully connected layer. If the accuracy is greater than the first threshold, the second stage of training is started. Otherwise, the number of nodes in the fully connected layer FC1 is increased, and a fully connected layer is added to the fully connected layer FC1 until the accuracy is greater than the first threshold.

[0084] The second stage of training involves using training sets from different stages of the same embryo for each Conv training session. i Training is performed with fully connected layers. When the accuracy exceeds the first threshold, the second stage of training is stopped. Otherwise, the training sets of the same embryo at each stage are enhanced in the same way, and then the enhanced training sets are used for training again.

[0085] Preferably, step four is included after step three, such as... Figure 6 As shown;

[0086] Step four: Receive the graded probability values ​​output by each sub-convolutional neural network, calculate the standard deviation of all probability values, and when the standard deviation is greater than the second threshold, display the graded information of the embryo while simultaneously displaying the image corresponding to the stage with the largest deviation from the average value of the probability values.

[0087] Preferably, setting up a sub-convolutional neural network for each stage specifically involves:

[0088] The visualization interface displays configuration items for configuring the sub-convolutional neural network. These configuration items include the stage name and the specific structure of the sub-neural network. The specific structure includes the order and number of convolutional layers, pooling layers, fully connected layers, and activation layers. The specific structure also includes the size and number of convolutional kernels in the convolutional layers, the number of fully connected layers, and the number of nodes.

[0089] Example 3

[0090] This invention also provides an intelligent method for assisting embryo grading, the method comprising the following steps:

[0091] Step 1: Obtain embryo images of livestock fusion cells at at least one stage from activation to transplantation. Each stage includes at least one embryo image. Construct an image set corresponding to each stage, put the preprocessed embryo images into the corresponding image set, and label the image set to obtain the training set, validation set, and test set for each stage.

[0092] Step 2: Set up a sub-convolutional neural network for each stage, train the sub-convolutional neural network using the training set of each stage, and adjust the hyperparameters using the validation set corresponding to each stage to obtain the trained sub-convolutional neural network for each stage; connect a convolutional layer Conv to the last convolutional layer of each trained sub-convolutional neural network. i For each convolutional layer Conv i The output is flattened and then subjected to feature fusion, serving as the input to the fully connected layer FC1. The structure and parameters of the last convolutional layer in each sub-convolutional neural network remain unchanged. Based on the training set, Conv... i Retrain the fully connected layer to obtain a trained neural network;

[0093] Step 3: Input the embryo image of the fused cells to be identified into the trained neural network to obtain the embryo grading information.

[0094] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of a necessary general-purpose hardware platform, or by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent system for assisting embryo grading, characterized in that, The system includes the following modules: The sample set construction module is used to acquire embryo images of livestock fusion cells from activation to at least one stage before transplantation. Each stage includes at least one embryo image. The module constructs an image set corresponding to each stage, puts the preprocessed embryo images into the corresponding image set, and annotates the image set to obtain the training set, validation set, and test set for each stage. The model building and training module is used to set up a sub-convolutional neural network for each stage, train the sub-convolutional neural network using the training set of each stage, and adjust the hyperparameters using the validation set corresponding to each stage to obtain the trained sub-convolutional neural network for each stage. A convolutional layer Conv is then connected to the last convolutional layer of each trained sub-convolutional neural network. i For each convolutional layer Conv i The output is flattened and then subjected to feature fusion, serving as the input to the fully connected layer FC1. The structure and parameters of the last convolutional layer in each sub-convolutional neural network remain unchanged. Based on the training set, Conv... i Retrain the fully connected layer to obtain a trained neural network; The livestock embryo grading module is used to input the embryo image of the fused cells to be identified into the trained neural network to obtain the grading information of the embryo; Convolutional layer Conv i The number and size of the convolutional kernels are determined based on the stage and the number of nodes in the fully connected layer FC1, specifically as follows: Obtain the stage number K and the number of nodes M in the fully connected layer FC1, and then determine the number of nodes allocated to the fully connected layer FC1 for each stage. α is the weight for assigning the number of FC1 nodes in the fully connected layer to each stage; Choose one or more from 1×1, 3×3, and 5×5 as the convolutional layer Conv. i The middle convolution kernel makes the convolutional layer Conv i The number of feature points in the output feature map and the number of nodes w i same; The Conv based on the training set i Retraining with the fully connected layer yields a trained neural network, specifically: Each time, training samples from the training set of each stage are selected and input into the sub-neural network corresponding to that stage, and Conv... i The first stage of training is performed with the fully connected layer. If the accuracy is greater than the first threshold, the second stage of training is started. Otherwise, the number of nodes in the fully connected layer FC1 is increased, and a fully connected layer is added to the fully connected layer FC1 until the accuracy is greater than the first threshold. The second stage of training involves using training sets from different stages of the same embryo for each Conv training session. i Training is performed with fully connected layers. When the accuracy is greater than the first threshold, the second stage of training is stopped. Otherwise, the training sets of each stage of the same embryo are enhanced in the same way, and the enhanced training sets are used for training again. The system also includes a calibration module; The correction module is used to receive the graded probability values ​​output by each sub-convolutional neural network, calculate the standard deviation of all probability values, and when the standard deviation is greater than a second threshold, while displaying the graded information of the embryo, it also displays the image corresponding to the stage with the largest deviation from the average value of the probability values.

2. The system as described in claim 1, characterized in that, The specific steps for setting up a sub-convolutional neural network for each stage are as follows: The visualization interface displays configuration items for configuring the sub-convolutional neural network. These configuration items include the stage name and the specific structure of the sub-neural network. The specific structure includes the order and number of convolutional layers, pooling layers, fully connected layers, and activation layers. The specific structure also includes the size and number of convolutional kernels in the convolutional layers, the number of fully connected layers, and the number of nodes.

3. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the following method: Step 1: Obtain embryo images of livestock fusion cells at at least one stage from activation to transplantation. Each stage includes at least one embryo image. Construct an image set corresponding to each stage, put the preprocessed embryo images into the corresponding image set, and label the image set to obtain the training set, validation set, and test set for each stage. Step 2: Set up a sub-convolutional neural network for each stage, train the sub-convolutional neural network using the training set of each stage, and adjust the hyperparameters using the validation set corresponding to each stage to obtain the trained sub-convolutional neural network for each stage; connect a convolutional layer Conv to the last convolutional layer of each trained sub-convolutional neural network. i For each convolutional layer Conv i The output is flattened and then subjected to feature fusion, serving as the input to the fully connected layer FC1. The structure and parameters of the last convolutional layer in each sub-convolutional neural network remain unchanged. Based on the training set, Conv... i Retrain the fully connected layer to obtain a trained neural network; Step 3: Input the embryo image of the fused cells to be identified into the trained neural network to obtain the embryo grading information; Convolutional layer Conv i The number and size of the convolutional kernels are determined based on the stage and the number of nodes in the fully connected layer FC1, specifically as follows: Obtain the stage number K and the number of nodes M in the fully connected layer FC1, and then determine the number of nodes allocated to the fully connected layer FC1 for each stage. α is the weight for assigning the number of FC1 nodes in the fully connected layer to each stage; Choose one or more from 1×1, 3×3, and 5×5 as the convolution kernel in the convolutional layer Convi, such that the convolutional layer Convi... i The number of feature points in the output feature map and the number of nodes w i same; The Conv based on the training set i Retraining with the fully connected layer yields a trained neural network, specifically: Each time, training samples from the training set of each stage are selected and input into the sub-neural network corresponding to the stage. The first stage of training is performed on Conv and the fully connected layer. When the accuracy is greater than the first threshold, the second stage of training is entered. Otherwise, the number of nodes in the fully connected layer FC1 is increased and a fully connected layer is added to the fully connected layer FC1 until the accuracy is greater than the first threshold. The second stage of training involves using training sets from different stages of the same embryo for each Conv training session. i Training is performed with fully connected layers. When the accuracy is greater than the first threshold, the second stage of training is stopped. Otherwise, the training sets of each stage of the same embryo are enhanced in the same way, and the enhanced training sets are used for training again. Step three is followed by step four; Step four: Receive the graded probability values ​​output by each sub-convolutional neural network, calculate the standard deviation of all probability values, and when the standard deviation is greater than the second threshold, display the graded information of the embryo while simultaneously displaying the image corresponding to the stage with the largest deviation from the average value of the probability values.

4. The readable storage medium as described in claim 3, characterized in that, The specific steps for setting up a sub-convolutional neural network for each stage are as follows: The visualization interface displays configuration items for configuring the sub-convolutional neural network. These configuration items include the stage name and the specific structure of the sub-neural network. The specific structure includes the order and number of convolutional layers, pooling layers, fully connected layers, and activation layers. The specific structure also includes the size and number of convolutional kernels in the convolutional layers, the number of fully connected layers, and the number of nodes.

Images