Portable recording medium, information processing device, information processing method, information display method, and method for manufacturing a learning model

A learning model-based sperm evaluation system efficiently selects sperm for microinsemination, addressing inefficiencies and damage in existing methods by predicting successful fertilization and embryo growth.

CN112889088BActive Publication Date: 2024-12-03HU GROUP RESEARCH INSTITUTE G K
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201980068885.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-10-31
Filing Date
2019-10-31
Publication Date
2024-12-03
Estimated Expiration
2039-10-31

AI Technical Summary

Technical Problem

Existing methods for sperm evaluation in microinsemination are time-consuming, inefficient, and prone to damaging sperm and eggs, and do not reliably select sperm with a high probability of successful fertilization and normal embryo growth.

Method used

A program recorded on a removable recording medium uses a learning model to analyze sperm images, predicting the success probability of microinsemination and normal embryo formation, enabling efficient and accurate sperm selection.

Benefits of technology

The solution supports the selection of sperm with a high probability of successful microinsemination and normal embryo formation, reducing time and effort while minimizing damage to sperm and eggs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112889088B_ABST
    Figure CN112889088B_ABST
Patent Text Reader

Abstract

The present invention provides a portable recording medium, an information processing device, an information processing method, an information display method, and a method for manufacturing a learning model. The program causes a computer to execute the following processing: obtaining a captured image, the captured image capturing a candidate sperm to be used for microinsemination, inputting the acquired captured image into a learning model (53), the learning model receiving the captured image capturing the sperm and outputting a prediction of whether microinsemination using the sperm is successful or not, and outputting the prediction output by the learning model (53) based on the input captured image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a portable recording medium, an information processing device, an information processing method, an information display method, and a method for manufacturing a learning model. Background Art

[0002] Assisted reproductive technology (ART) is used as a treatment for infertility. One of the ARTs used clinically is microinsemination, which involves injecting sperm into an egg for fertilization under a microscope.

[0003] Microinsemination is performed by an assisted reproductive technology embryologist or clinical embryologist (hereinafter referred to as an embryologist) who injects sperm selected visually into the egg under a microscope for fertilization. In this case, ICSI (Intracytoplasmic Sperm Injection) or IMSI (Intracytoplasmic Morphologically selected Sperm Injection) is performed.

[0004] In ICSI, the incubator uses a microscope with a magnification of 200 to 400 times to observe and extract sperm. In IMSI, the incubator uses a microscope with a magnification of more than 1,000 times to observe and extract sperm in detail.

[0005] Microinsemination is expected to be a particularly effective treatment for male infertility such as oligospermia, azoospermia, and asthenozoospermia, because it is possible to obtain a fertilized egg from one sperm and one egg. However, a method for reliably obtaining a normal fertilized egg through microinsemination and a method for reliably obtaining normal embryo growth after fertilization have not yet been established.

[0006] Microinsemination can obtain fertilized eggs more efficiently than in vitro fertilization, and can obtain a high implantation rate or pregnancy rate, or a low miscarriage rate compared to natural pregnancy and artificial insemination (Non-Patent Document 1). Therefore, microinsemination has been flexibly used as a treatment for infertility not limited to male infertility.

[0007] It is known that sperm with abnormal morphology or motility are not suitable for fertilization due to genetic problems such as sperm maturity or DNA (deoxyribonucleic acid) fragmentation, and also have problems with embryo growth after fertilization. Therefore, a method for detecting sperm morphology abnormalities has been proposed (Patent Document 1).

[0008] In addition, the laboratory manual on sperm evaluation methods published by WHO (World Health Organization) (non-patent document 2) also introduces an index for statistically evaluating multiple sperm contained in the semen of sperm donors. According to this index, when the proportion of sperm with normal morphology is less than 4%, the success rate (also referred to as success probability or predicted success probability) of assisted reproductive technology is low.

[0009] Based on this background, among the many devices sold on the market, there is a simple semen quality evaluation method implemented when performing infertility treatment such as microinsemination, which performs a simple evaluation of multiple sperm in semen at a low resolution and numerically evaluates the proportion of sperm in which abnormalities cannot be detected (for example, confirming whether the proportion of sperm with normal shape is less than 4%). In addition, a technology for objectively selecting sperm by evaluating the shape of sperm using a rule base has been proposed (Non-Patent Document 3).

[0010] Prior art literature

[0011] Patent Literature

[0012] Patent Document 1: Korean Patent Publication No. 2011-0049606

[0013] Non-patent literature

[0014] Non-patent document 1: A. Hazout et al., "High-magnification ICSI overcomes spaternal effect resistant to conventional ICSI", Reproductive Biomedicine Online, VOL. 12, NO. 1, pp19-25, 2006

[0015] Non-patent document 2: World Health Organization, Department of Reproductive Health and Research, "WHO laboratory manual for the examination and processing of human semen Fifth edition", 2010

[0016] Non-patent literature 3: V. Chang et al, "Automatic classification of human spermhead morphology", Computers in Biology and Medicine, VOL. 84, pp205-216, 2017. Summary of the invention

[0017] Problems to be solved by the invention

[0018] However, the methods disclosed in Patent Document 1 and Non-Patent Documents 1 to 3 cannot be applied to microinsemination, which requires sperms to be evaluated one by one. Even if it can be applied technically, sperm evaluation takes a lot of time and effort, and the damage to sperms and eggs will increase during the process, so it is not practical. In addition, it cannot be used to support the selection of sperms with a high probability of normal embryo growth after microinsemination and fertilization from sperms with no abnormal morphology or motility.

[0019] An object of the present invention is to provide a portable recording medium or the like that supports the selection of sperm with a high probability of successful microinsemination or the selection of sperm with a high probability of normal embryo formation after insemination.

[0020] Means for solving problems

[0021] A program is recorded in a removable recording medium, and the program causes a computer to execute the following processing: obtaining a captured image, the captured image capturing a candidate sperm to be used in microinsemination, inputting the acquired captured image into a learning model, the learning model accepting the captured image capturing the sperm and outputting a prediction related to the success or failure of microinsemination using the sperm, and outputting a prediction result output by the learning model based on the input captured image.

[0022] Effects of the Invention

[0023] One effect of the present invention is to provide a portable recording medium or the like that supports the selection of sperm with a high probability of successful microinsemination or the selection of sperm with a high probability of normal embryo formation after insemination. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is an explanatory diagram for explaining the processing flow using the information processing system.

[0025] Figure 2 This is an explanatory diagram for explaining the structure of an information processing system.

[0026] Figure 3 This is an illustration that explains the process until delivery.

[0027] Figure 4 This is an explanatory diagram that illustrates the frequency distribution of success probability.

[0028] Figure 5 This is an explanatory diagram for explaining a method for creating an image encoder that extracts image feature quantities from a still image obtained by photographing sperm.

[0029] Figure 6 This is an explanatory diagram for explaining the extraction of image feature quantities from a moving image file.

[0030] Figure 7 This is an illustration of the process learning model.

[0031] Figure 8 It is an explanatory diagram for explaining the record structure of the training data DB.

[0032] Fig. 9 It is an explanatory diagram for explaining the record structure of the preliminary shooting DB.

[0033] Fig.10 This is an explanatory diagram showing a screen displayed by the information processing system.

[0034] Fig.11 This is an explanatory diagram showing a screen displayed by the information processing system.

[0035] Fig.12 This is an explanatory diagram showing a screen displayed by the information processing system.

[0036] Fig.13 This is a flowchart showing the processing flow of the program used in the first preparation stage.

[0037] Fig.14 : is a flowchart showing the processing flow of the program used in the second preparation stage and the sperm selection stage.

[0038] Fig.15 This is a flowchart showing the processing flow of the sampling distribution calculation subroutine.

[0039] Fig.16 This is an explanatory diagram for explaining the record structure of the preliminary shot DB according to the second embodiment.

[0040] Fig.17 This is a flowchart showing the processing flow of the sampling distribution calculation subroutine in the second embodiment.

[0041] Fig.18 This is an explanatory diagram showing a screen displayed by the information processing device according to the third embodiment.

[0042] Fig.19 This is an explanatory diagram for explaining the structure of the information processing system according to the fourth embodiment.

[0043] Fig. 20 It is an explanatory diagram for explaining the record structure of the normal sperm determination DB.

[0044] Fig.21 It is an explanatory diagram showing the structure of a normal sperm determination model.

[0045] Fig. 22 This is an explanatory diagram showing a screen displayed by the information processing device according to the fourth embodiment.

[0046] Fig.23 This is an explanatory diagram showing a screen displayed by the information processing device according to the fourth embodiment.

[0047] Fig.24 This is an explanatory diagram showing a screen displayed by the information processing device according to the fourth embodiment.

[0048] Fig.25 This is a flowchart showing the processing flow of the program of implementation mode 4.

[0049] Fig.26 This is a flowchart showing the processing flow of the sperm image acquisition subroutine.

[0050] Fig. 27 This is an explanatory diagram showing a screen displayed by the information processing device according to the fifth embodiment.

[0051] Fig.28 This is a functional block diagram of an information processing device according to a sixth embodiment.

[0052] Fig.29 This is an explanatory diagram showing the structure of an information processing system according to a seventh embodiment.

[0053] Fig.30 It is a table showing the incidence and carrier rate of sex-linked inferior (recessive) genetic diseases.

[0054] Fig.31 It is a table showing the incidence and carrier rate of sex-linked dominant genetic diseases.

[0055] Fig.32 This is a table showing the incidence and carrier rate of sex-linked genetic diseases inherited through the Y chromosome.

[0056] Fig.33 It is an explanatory diagram for explaining the record structure of the gender training data DB.

[0057] Fig.34 This is an explanatory diagram for explaining the gender determination learning model.

[0058] Fig.35 This is an explanatory diagram showing a screen displayed by an information processing device according to the eighth embodiment.

[0059] Fig.36 It is a flowchart showing the processing flow of the program of implementation mode 8.

[0060] Fig.37 This is an explanatory diagram showing a screen displayed by an information processing device according to the ninth embodiment.

[0061] Fig.38 This is an explanatory diagram showing a screen displayed by an information processing device according to the ninth embodiment.

[0062] Fig.39 It is a flowchart showing the processing flow of the program of implementation mode 9.

[0063] Fig.40 1 is a flowchart showing the processing flow of the incidence rate and carrier rate calculation subroutine. DETAILED DESCRIPTION

[0064] [Implementation Method 1]

[0065] Figure 1 It is an explanatory diagram for explaining the flow of processing using the information processing system 10 . Figure 1 A represents the first preparation stage in which information related to past microinseminations is processed.

[0066] As described above, in microinsemination, the incubator uses a microscope 41 with an optical magnification of 200 to 400 times, or 1000 times or more (see Figure 2 ) Observe the sperm in the semen. The incubator selects a sperm with a normal shape and normal movement for each egg. In addition, the incubator can also observe the image displayed on the display device 15 after the camera 48, the shooting I / F 28 or the display I / F 25 uses a digital magnification to magnify the microscope image several times.

[0067] The incubator takes a microscope image, which includes the sperm selected by a moving image or a still image. The microscope image may be cut out by image processing or the like to capture the portion of the sperm. In the following description, the captured microscope image may be referred to as a captured image. The incubator extracts the selected sperm using a micropipette.

[0068] The incubator inserts the tip of the micropipette into the egg and injects the selected sperm. This is microinsemination. Then, the incubator cultivates the egg under specified conditions. The egg that has been fertilized and started to grow is called an embryo. The obstetrician transplants the embryo that has grown to a specified stage, such as a 4-cell embryo or blastocyst, into the mother's body. After the embryo implants in the uterus, the pregnancy is completed. After that, the fetus grows in the uterus like a normal natural pregnancy, and as long as it is normal, it can be delivered.

[0069] The time course related to the growth of the embryo after microinsemination, such as whether fertilization is completed, whether normal growth to each growth stage such as 4-cell embryo, morula, blastocyst, etc., whether pregnancy is completed, whether abnormalities are found in pregnancy tests, whether delivery is successful, etc., are recorded in association with the captured images.

[0070] The image feature quantity and motion feature quantity (hereinafter collectively referred to as feature quantity) extracted from the image taken when selecting sperm, the clinical personal information of the sperm donor (age, past medical history, current medical history, treatment history, and genomic information such as gene mutation, etc.), and the growth process of the fertilized egg or embryo after microinsemination are recorded in the training data DB 51 in association with each other. The training data DB 51 will be described in detail later.

[0071] A supervised machine learning is performed based on the training data DB51 to create a machine learning model (hereinafter referred to as a "process learning model") 53 for predicting the growth process of a fertilized egg or embryo. The process learning model 53 is a learned model that outputs the success probability of normal growth at each stage in the process from fertilization to delivery when the feature quantity of the captured image of the sperm is input. The feature quantity and the process learning model 53 will be described in detail later.

[0072] The probability of failure can be easily calculated by subtracting the probability of success from 1. Therefore, the above probability of success can also be replaced by the probability of failure. This applies to all subsequent success probabilities.

[0073] Based on the frequency distribution of the data on success or failure of each stage from fertilization to delivery and the normalization of the frequency distribution, a statistical analysis is performed to create Figure 1 Bottom right and Figure 4 The first distribution f(X) is shown in the graph as a solid line. Figure 1 Bottom right and Figure 4 The curve graph shown is made into one for each stage. The first distribution f(X) represents the probability density distribution of the success probability of normal growth to each stage, which is equivalent to the so-called prior probability distribution in Bayesian inference. The first distribution f(X) will be described in detail later.

[0074] Figure 1 In the first preparation stage shown in A, sperm images taken for a plurality of cases and data on the growth process of fertilized eggs or embryos are used. The process learning model 53 and the first distribution f(X) created in the first preparation stage can be used for cases to be implemented later.

[0075] Figure 1 B and Figure 1 C represents the second preparation stage and the sperm selection stage respectively implemented for the cases newly subjected to microinsemination. Preferably, the incubator implements the second preparation stage and the sperm selection stage continuously after receiving the semen extracted from the sperm donor.

[0076] Figure 1 In the second preparation stage shown in B, a sampling distribution g(X) is created based on the characteristics of sperm contained in the semen extracted from the sperm donor. The culture technician performs preparatory shooting of sperm after pre-processing such as washing and diluting the semen. By inputting the feature quantity extracted from the shot image into the process learning model 53 for predicting the growth process of the fertilized egg or embryo, the probability of normal growth related to the sperm at each stage from fertilization to delivery (hereinafter referred to as "prediction success probability") can be obtained.

[0077] Based on the predicted success probability of multiple sperm, Figure 1 Bottom right and Figure 4 The sampling distribution g(X) is shown as a dotted line in the graph. The sampling distribution g(X) represents the distribution of the predicted success probability of sampling and evaluating the sperm contained in the semen of the sperm donor, which is equivalent to the so-called likelihood distribution in Bayesian inference. The sampling distribution g(X) will be described in detail later.

[0078] Based on the first distribution f(X) and the sampling distribution g(X), a second distribution h(X) is generated. The second distribution h(X) is equivalent to the so-called posterior probability distribution in Bayesian inference. The second distribution h(X) will be described in detail later.

[0079] Figure 1 In the sperm selection stage shown in C, sperm to be used for microinsemination is selected from the semen of the sperm donor. The incubator takes an image of the candidate sperm. By inputting the feature quantity extracted from the captured image into the process learning model 53, the predicted success probability of each stage from normal growth to fertilization to delivery related to the candidate sperm can be obtained.

[0080] The position of the predicted success probability within the second distribution h(X) is displayed, for example, by an evaluation index such as a deviation value. The culture technician can use the displayed evaluation index to determine whether to use the candidate sperm for microinsemination. If it is determined to be used for microinsemination, the culture technician extracts the candidate sperm under microscope observation and injects it into the egg. If it is determined not to be used for microinsemination, the culture technician evaluates other candidate sperm.

[0081] In this way, the incubator can select sperm with a high probability of successful microinsemination from the extracted semen. In the following description, sperm that meets the specified criteria for evaluation indicators at each stage, that is, sperm with a high enough probability of successful microinsemination and normal delivery, is recorded as "high-quality sperm".

[0082] Normal spermatozoa move at high speed in a short time in a solution by flagellar movement. When performing microinsemination, observation conditions such as the viscosity of the solution and the sperm concentration in the solution are adjusted to slow down the movement of the spermatozoa. Thus, the incubator can fully observe the spermatozoa. However, when the spermatozoa move out of the field of view of the microscope 41, it is difficult to track or find the same spermatozoa.

[0083] Therefore, in the conventional method in which the incubator judges the quality of sperm by microscopic observation, even if the incubator compares and studies multiple sperm, it is difficult to ensure that the sperm used for microinsemination is the same sperm. In the present embodiment, since the evaluation of each candidate sperm can be displayed in real time, the incubator can easily judge whether to use the candidate sperm and extract the candidate sperm as the object.

[0084] When selecting sperm, when the semen contains a large number of high-quality sperm, sperm is extracted at the time when such sperm is obtained, or at the time when better indicators are obtained according to the second distribution h(X), and the process is switched to microinsemination. This can improve the work efficiency of the incubator and minimize the damage to sperm and eggs.

[0085] Even when there are few sperms suitable for microinsemination in semen, it is preferable to select the best quality sperms within a limited time. For example, a method can be considered in which sperms with relatively good evaluation results are captured and taken out for storage each time, and the best quality sperms among the stored sperms are used in the final stage. As the method of taking out and storing, there are a method of sucking specific sperms with a capillary tube and transferring them to a microplate, and a method of separately holding a droplet containing specific sperms in a solution-free area on a slide.

[0086] On the other hand, after giving up the selection of the best quality sperm due to time constraints and labor constraints, it is preferred to adopt a method that has the highest probability of selecting the best quality sperm.

[0087] In other fields, the mathematical optimal solution is known, that is, to evaluate the candidates one by one and end the evaluation when a good candidate is found. This is a type of optimal stopping problem, known as the secretary problem. The optimal solution calculation process for the secretary problem is explained. When the total number of candidates (the total number of sperm in the semen) n is a sufficiently large value, the initial n / e (e is a natural constant) candidates are only evaluated without selection. For the candidates (candidate sperm) evaluated later, if a candidate is judged to be better than all the candidates evaluated previously, the candidate is determined to be the best candidate.

[0088] More specifically, since 1 / e is equivalent to about 37%, about 37% of all sperms contained in the semen are not used after the evaluation value is confirmed. Among the remaining sperms, if a sperm with a better evaluation value than all the sperms that have been evaluated is obtained, the sperm is used.

[0089] When using the optimal solution to the secretary problem, there is no need to take out and store sperm, and statistically optimal work efficiency can be expected. On the other hand, even if the optimal solution to the secretary problem is used, it is not always possible to select sperm with the highest evaluation value contained in semen. For example, the following two situations can be imagined. One situation is that the optimal candidate is included in the initial approximately 37% of candidates. Another situation is that the quality of the initial approximately 37% of candidates is very low, and although a high-quality candidate is found among the subsequent candidates, there are still candidates with better quality among the unevaluated candidates.

[0090] Although the frequency of occurrence of these two situations is low in statistics, it is still desirable to avoid such phenomena. To this end, it is desirable to select sperm based on the deviation value, reliability and other indicators obtained from the second distribution h(X). In this case, compared with the optimal solution to the secretary problem, the statistical efficiency is low, but candidates with good quality can be selected.

[0091] In addition, when the number of sperm in semen is very large, it is not realistic to observe about 37% of the sperm. Therefore, it is a realistic method to select candidates based on the second distribution h(X). In the case of completely ignoring the work efficiency, sperm with better quality can be obtained and stored, and after evaluating all sperm, the sperm with the best quality can be used from the stored sperm.

[0092] It is known that even for the same sperm donor, the characteristics of sperm count and characteristic amount contained in semen vary depending on the period of semen extraction. Figure 1 B. Second stage of preparation and use of instructions Figure 1 The sperm selection stage described in C is preferably performed continuously using the same semen. However, in the case where the sperm count is extremely low due to oligospermia or the like, semen extracted from the same sperm donor at different times may be used in the second preparation stage and the sperm selection stage.

[0093] In addition, when the sperm donor has symptoms such as azoospermia in which sperm is not contained in the semen, sperm is extracted by surgery such as epididymal sperm aspiration or testicular sperm aspiration instead of extracting semen. When there are few sperms that can be extracted, it is sometimes difficult to implement the second preparation stage and the sperm selection stage. In such a case, the second preparation stage can be omitted and the sperm selection stage can be implemented using the sampling distribution g(X) of sperm donors with similar symptoms.

[0094] Figure 21 is an explanatory diagram for explaining the structure of the information processing system 10. The information processing system 10 includes an information processing device 20, a display device 15, and a microscope 41.

[0095] The information processing device 20 includes a control unit 21, a main storage device 22, an auxiliary storage device 23, a communication unit 24, a display I / F (Interface) 25, a shooting I / F 28, and a bus. The control unit 21 is an operation control device that runs the program of this embodiment. The control unit 21 is composed of one or more CPUs (Central Processing Units), multi-core CPUs, or GPUs (Graphics Processing Units). The control unit 21 is connected to each hardware component of the information processing device 20 through a bus.

[0096] The main storage device 22 is a storage device such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), flash memory, etc. The main storage device 22 can temporarily store information required in the process executed by the control unit 21 and the program running on the control unit 21.

[0097] The auxiliary storage device 23 is a storage device such as SRAM, flash memory or hard disk. The auxiliary storage device 23 can store the training data DB 51, preliminary shot DB 52, process learning model 53, image encoder 546, programs executed by the control unit 21, and various data required for program execution.

[0098] In addition, the training data DB 51 , the preliminary shot DB 52 , the process learning model 53 , and the image encoder 546 may also be stored in an external large-capacity storage device or the like connected to the information processing device 20 .

[0099] The communication unit 24 is an interface for communicating between the information processing device 20 and the network. The display I / F 25 is an interface for connecting the display device 15 such as a liquid crystal display device or an organic EL (Electro Luminescence) display device and the information processing device 20. The shooting I / F 28 is an interface for connecting the camera 48 described later and the information processing device 20.

[0100] The display I / F 25 is, for example, a VGA terminal, a DVI (Digital Visual Interface) terminal, an HDMI (registered trademark) (High-Definition Multimedia Interface) terminal, or a USB (Universal Serial Bus) terminal. The photographing I / F 28 is, for example, a USB terminal. The display I / F 25 and the display device 15, and the photographing I / F 28 and the camera 48 may be connected wirelessly.

[0101] The microscope 41 is, for example, a differential interference microscope, a bright field microscope, a polarizing microscope, a phase contrast microscope, or an inverted microscope. The microscope 41 includes a stage 42, an eyepiece 43, an objective lens 47, and an illumination unit 44. An observation container 421 can be placed on the stage 42, and the observation container 421 contains semen that has been pre-treated by washing and dilution. The observation container 421 is, for example, a shallow bottom plate, a well plate, or a slide.

[0102] The observation container 421 is illuminated by the illumination light emitted by the illumination unit 44. The culture technician as a user observes the sperm in the observation container 421 through the objective lens 47 and the eyepiece 43.

[0103] An optical path dividing unit 45 is disposed between the objective lens 47 and the eyepiece 43. The sperm that the culture technician is observing can be photographed as a moving image or a still image by using a camera 48 connected to the optical path dividing unit 45. The photographed image is recorded in the auxiliary storage device 23 through the photographing I / F 28, and is displayed in real time on the display device 15 through the display I / F 25.

[0104] For example, when searching for candidate sperms for confirmation and evaluation, the trainer observes the sperms displayed on the display device 15. When extracting candidate sperms with a micropipette, the trainer visually confirms the candidate sperms and the tip of the micropipette using the eyepiece 43. In this way, since the trainer can flexibly use the display device 15 and the eyepiece 43, it is possible to provide an information processing system 10 that reduces the trainer's fatigue and enables accurate sperm extraction.

[0105] The information processing device 20 of this embodiment is a general-purpose personal computer, tablet computer, mainframe computer, or a virtual machine operating on a mainframe computer. The information processing device 20 may also be composed of hardware such as multiple personal computers, tablet computers, or mainframe computers. The information processing device 20 may also be composed of a quantum computer. The information processing device 20 may also be built into the microscope 41. The information processing device 20 may also be connected to the network through a hospital network system (not shown).

[0106] Figure 3This is an illustration of the process leading to childbirth. Several to dozens of eggs are extracted from egg donors. Among the extracted eggs, those with abnormalities such as deformation and eggs that are not yet mature enough for fertilization are discarded. Tens of thousands to tens of millions of sperm are extracted from sperm donors. Sperm are selected and injected into eggs for microinsemination. Eggs are cultured under specified conditions.

[0107] After successful fertilization, the embryo begins cell division, undergoes 2-cell embryos, 4-cell embryos, etc., and grows into morulas and blastocysts. The growth process of the embryo is appropriately observed, and whether it grows normally to each growth stage is recorded. Abnormal embryos found through observation are discarded.

[0108] Embryos that have grown to a certain stage are transplanted into the mother, and the remaining embryos are cryopreserved. In addition, there are also cases where frozen embryo transplantation is performed, that is, all embryos are temporarily frozen and stored, and then the embryos are transplanted when the mother is in good condition.

[0109] After the embryo implants in the uterus and pregnancy is successful, the fetus grows in the same way as a natural pregnancy. Pregnancy check-ups are performed and the fetus's growth process is recorded. If the process is normal, delivery will occur and a newborn will be born.

[0110] In cases where the embryo does not implant in the uterus, in the case of miscarriage, or in the case of the desire to have a second or subsequent child, the cryopreserved embryo can be used for another embryo transfer. After the infertility treatment is completed, the cryopreserved embryo is discarded.

[0111] Figure 4 This is an explanatory diagram for explaining the distribution of success probability. The horizontal axis X represents the success probability as a probability variable, and the vertical axis Y represents the probability corresponding to the probability variable X. When the success probability is 0, it means that there is no success case, and when the success probability is 1, it means that all cases are successful.

[0112] Figure 1 The graph shown in the lower right also shows Figure 4 The same distribution, but the vertical axis X represents the probability of success as a probability variable, and the horizontal axis Y represents the probability corresponding to the probability variable X.

[0113] The success probability is calculated for each stage of fertilization, blastocyst formation, implantation, and delivery. For example, the success probability of fertilization and blastocyst formation is calculated by the ratio of the number of eggs that grow normally to fertilization and blastocyst formation to the number of eggs that have undergone microinsemination. The success probability of implantation and delivery is calculated by the ratio of the number of embryos that have successfully implanted and delivered to the number of embryos that have been transferred to the mother.

[0114] The success probability can also be calculated for various stages of development such as 2-cell embryos, 4-cell embryos, morulae, early pregnancy, mid-pregnancy, etc. The success probability of normal growth of infants and young children after delivery to various growth stages can also be calculated.

[0115] Each stage is evaluated as either normal or not. The first distribution f(X) shown by the solid line is the probability density distribution of success or failure of all recorded cases expressed by the Beta distribution. The first distribution f(X) is represented by formula (1).

[0116]

Number 1

[0117] Y=f(X)=C1X p-1 (1-X) q-1 ……(1)

[0118] p represents the number of cases that grow normally to each growth stage.

[0119] q represents the number of cases that did not grow normally to each growth stage.

[0120] C1 represents a normalization constant.

[0121] The dotted line shows the sampling distribution g(X) when using Figure 1 B describes the distribution of the success probability predicted by the preparatory images in the second preparation stage. For example, when a binary prediction of whether the animal can grow normally to each growth stage is made based on the preparatory images, the sampling distribution g(X) is expressed by equation (2) based on the binomial distribution.

[0122]

Number 2

[0123]

[0124] a represents the number of preliminary captured images predicted to be normal growth at each growth stage.

[0125] b represents the number of preliminary captured images predicted to be unable to grow normally to each growth stage.

[0126] C2 represents a normalization constant.

[0127] The second distribution h(X) indicated by the single-point dashed line is a posterior probability distribution obtained by Bayesian inference using the first distribution f(X) as the prior probability distribution and the sampling distribution g(X) as the likelihood distribution, and represents the result of predicting the probability density distribution of the success probability of the sperm provided by the sperm donor. The second distribution h(X) is represented by formula (3).

[0128]

Number 3

[0129] Y=h(X)=C3f(X)g(X)……(3)

[0130] C3 is the normalization constant.

[0131] The closer the success probability calculated by the process learning model 53 for each candidate sperm is to the right end of the second distribution h(X), the higher the success probability of the sperm among the provided sperm. Figure 4 The sperm with a success probability of about 0.55 shown in A is a sperm with a relatively low success probability in all the data represented by the first distribution f(X), but it is a sperm with a relatively high success probability in the provided sperm represented by the second distribution h(X). Based on this information, the culture technician determines whether to use the sperm under observation for microinsemination or to search for sperm with a higher success probability.

[0132] Figure 5 This is an explanatory diagram for explaining a method of creating an image encoder 546 for extracting image feature quantities from a still image of sperm.

[0133] use Figure 5 The image feature extraction method using the autoencoder is described below: The image feature model 54 is a CNN (Convolution Neural Network) including an input layer 541 , an intermediate layer 542 , and an output layer 543 .

[0134] CNN is a neural network that uses a ReLU (Rectified Linear Unit) function or a normalized exponential function as an activation function, and repeats a convolution layer and a pooling layer before repeating a fully connected layer multiple times. The convolution layer and the pooling layer are omitted in the illustration.

[0135] The input layer 541 and the output layer 543 have the same number of neurons as the number of pixels of the captured image as training data. The number of neurons in the intermediate layer 542 is the smallest in the central layer 545. A still image of the captured image is input to the input layer 541. Specifically, the pixel value of each pixel of the captured image is input to each neuron in the input layer 541.

[0136] The control unit 21 performs supervised machine learning to calculate parameters of the intermediate layer 542 using a back propagation algorithm or the like so that the output layer 543 outputs the same captured image as the input layer 541. Each neuron of the central layer 545 after the completion of the supervised machine learning represents the image feature value of the captured image.

[0137] Before performing supervised machine learning, the control unit 21 divides the captured images as training data into training data and verification data. The control unit 21 uses the verification data to verify the accuracy of the image feature quantity model 54 that has been subjected to supervised machine learning using the training data. This makes it possible to confirm that the image feature quantity model 54 has not been overtrained or has other problems. The learning and verification process is a process that is usually performed in supervised machine learning, and therefore, the description thereof will be omitted in the supervised machine learning described later.

[0138] The portion from the input layer 541 to the central layer 545 of the learned image feature quantity model 54 is used as an image encoder 546 for extracting image feature quantities from the captured image. The control unit 21 extracts the image encoder 546 from the image feature quantity model 54 and records it in the auxiliary storage device 23. Thus, the image encoder 546 that outputs image feature quantities when the captured image of sperm is input is completed.

[0139] It is preferable to use images of spermatozoa judged to be normal by experts such as skilled spermatozoa as training data when creating the image encoder 546. In this way, the image encoder 546 can be realized which extracts feature quantities that distinguish between normal spermatozoa.

[0140] When preparing training data for the image encoder 546, it is preferred that only one sperm is captured. When using an image in which multiple sperm or foreign matter is captured, the image is processed by cropping or the like to exclude portions other than the target sperm and used as training data.

[0141] The image encoder 546 may be created using any computer. The created image encoder 546 is sent to the information processing device 20 used by the microinsemination implementation mechanism via a network or the like, and is recorded in the auxiliary storage device 23. In this case, it is preferable that the information processing system 10 that creates the image encoder 546 and the information processing system 10 that uses the image encoder 546 use the microscope 41 of the same specification or a similar specification.

[0142] Figure 6 This is a diagram for explaining how to extract image features from a moving image file. Figure 6 The following describes the extraction of image feature quantities when a captured image during microinsemination is recorded as a moving image.

[0143] The following description will be made by taking the case where the moving images from time t1 to time t5 are recorded in the moving image file as an example. The moving image file is frame-divided to create a plurality of still images. Each still image is input into a pre-prepared judgment CNN to judge whether the whole sperm is captured. When the judgment CNN judges that the whole sperm is captured in a plurality of frames, one frame is selected by an arbitrary method.

[0144] exist Figure 6 In the example shown, it is determined that the entire sperm is captured in the frame at time t3. The still image determined to be the image of the entire sperm is input for use. Figure 5 In the image encoder 546 described above, feature quantities of a still image are extracted.

[0145] The moving image features such as the sperm's moving speed, the straight-line moving distance, and the tail flagella movement period are extracted from the moving image file. The moving image features are calculated based on, for example, the movement of multiple feature points defined on the captured image between frames. The moving image features can also be extracted using RNN (Recurrent Neural Network). In addition, any existing moving image analysis method can be used to extract the moving image features. The still image features and the moving image features are recorded in the training data DB 51 in association with the process after microinsemination.

[0146] When the image captured during microinsemination is recorded in both the moving image and still image formats, the control unit 21 does not cut out frames from the moving image file, but instead extracts image feature quantities using the still image.

[0147] Figure 7 53 is an explanatory diagram for explaining the process learning model 53. The process learning model 53 is a neural network including an input layer 531, an intermediate layer 532, and an output layer 533. Figure 7 , the case where the process learning model 53 is a CNN is illustrated. The convolution layer and the pooling layer are omitted in the illustration.

[0148] The process learning model 53 is created for each stage of fertilization, blastocyst formation, implantation, and delivery. The input of the process learning model 53 is the image feature quantity of the captured image, that is, the image feature quantity of the captured image. Figure 5 Image feature quantity of still image and use of the description Figure 6 The output of the process learning model 53 is the probability of normal growth to each stage (success probability) and the probability of failure to grow normally to each stage (failure probability).

[0149] When the feature quantities of still images and moving images are input to the input layer 531, the process learning model 53 outputs the probability of success and failure to the output layer 533. In the learning stage, the control unit 21 uses the training data DB 51 in which the feature quantities of still images and moving images are recorded in association with the process after microinsemination, and performs supervised machine learning by calculating the parameters of the intermediate layer 532 using a back propagation algorithm or the like.

[0150] Supervised machine learning can be performed using any method such as logistic regression, SVM (Support Vector Machine), random forest, CNN, RNN, or XGBoost (eXtreme Gradient Boosting).

[0151] The input layer 531 may be input with only one of the still image feature or the moving image feature. The input layer 531 may be additionally input with items other than the image feature, such as the age of the sperm donor and the egg donor, the past medical history, the current medical history including the health status, the family history, the past infertility treatment experience, etc. The input layer 531 may also be input with items related to sperm quality, such as the sperm concentration, the total sperm count, the sperm motility rate, the motile sperm concentration, the linear velocity, the curvilinear velocity, the average velocity, the linearity, the forwardness, the head side swing amplitude, and the whipping frequency, etc., obtained by low-magnification microscope observation or commercially available sperm evaluation equipment.

[0152] For example, after the learning of the process learning model 53 for determining whether fertilization is successful or not is completed, the control unit 21 may create a process learning model 53 for determining whether the subsequent stages such as the blastocyst stage are successful or not through transfer learning. The transfer learning utilizes the completed process learning model 53. From fertilization to delivery, the number of training data decreases as the stages become later. However, by utilizing transfer learning, an appropriate process learning model 53 can be created for the subsequent stages.

[0153] The process learning model 53 may be created by any computer. The created process learning model 53 is sent to the information processing device 20 used by the microinsemination implementation mechanism via a network or the like, and is recorded in the auxiliary storage device 23. In this case, it is preferable to use a microscope 41 of the same specification or a similar specification in the information processing system 10 that creates the image encoder 546 and the process learning model 53 and the information processing system 10 that uses the image encoder 546 and the process learning model 53.

[0154] Figure 8 : is an explanatory diagram for explaining the record structure of the training data DB 51. The training data DB 51 is a DB that records images of sperm used for microinsemination and the process after microinsemination in association with each other. The training data DB 51 has a sperm ID field, an image data field, a sperm image feature value field, and a process information field.

[0155] The sperm image feature field has a still image field and a moving image field. The still image field and the moving image field have a first feature field, a second feature field, etc. respectively. The process information field has a fertilization field, a blastocyst formation field, an implantation field, a delivery field, and a health status field. The training data DB51 has one record for one sperm used in microinsemination.

[0156] The sperm ID field records a unique sperm ID assigned to the sperm, and the image data field records image data of the sperm. Figure 8 Although an example in which a moving picture file with an extension of "mpg" is recorded is shown, the format of the image data is arbitrary. The image data field may also record a plurality of files such as a moving picture file and a still picture file.

[0157] Each subfield of the still picture field records the feature amount of the still picture obtained by inputting the image data into the image encoder 546. Each subfield of the moving picture field records the feature amount of the moving picture obtained by analyzing the image data.

[0158] The fertilization field records whether fertilization is successful or not, the blastocyst formation field records whether blastocyst formation is successful or not, the implantation field records whether implantation is successful or not, and the delivery field records whether delivery is successful or not. "OK" means that each stage has been reached normally, and "NG" means that each stage has not been reached normally. "-" means that the success or failure of each stage cannot be determined.

[0159] The health status field indicates the health status of the newborn. "OK" indicates that the newborn is healthy. "NG" indicates that the newborn has health problems, such as an extremely premature baby.

[0160] The process information field may have a field for recording any growth stage of the embryo, for example, a 2-cell embryo field, a 4-cell embryo field, a morula field, etc. The process information field may also have a field for recording the results of a pregnancy test, for example, a field for the twentieth week of pregnancy, etc. The process information field may also have a field for recording prenatal diagnosis results such as amniotic fluid examination or NIPT (Non-Invasive Prenatal Genetic Testing: prenatal screening of fetal free DNA in maternal peripheral blood).

[0161] The information recorded in each subfield of the process information field may be acquired from an electronic medical record or the like based on the sperm ID or the like recorded in the sperm ID field.

[0162] Fig. 9 is an explanatory diagram for explaining the record structure of the preliminary shooting DB 52. The preliminary shooting DB 52 is used Figure 1 The second preparation stage described in B is a DB that records the prediction results using the process learning model 53. The preparation shooting DB 52 has a patient ID field, a shooting date field, and a prediction result field. The prediction result field has a fertilization field, a blastocyst formation field, an implantation field, a delivery field, and a health status field. The preparation shooting DB 52 has one field for one preparation shooting.

[0163] The patient ID field records the unique patient ID assigned to the sperm donor. The shooting date field records the shooting date of the image. Each subfield of the prediction result field records the prediction result of whether each stage is reached normally. "OK" means that each stage is predicted to be reached normally, and "NG" means that each stage is not predicted to be reached normally.

[0164] The control unit 21 extracts sperm image features from the image captured by the camera 48, inputs the process learning model 53 created for each stage, and obtains the success probability. The control unit 21 records "OK" when the success probability is greater than a predetermined threshold, and records "NG" when it is less than the predetermined threshold.

[0165] Figures 10 to 12 It is an explanatory diagram showing a screen displayed by the information processing system 10 . Fig.10 Shown in use Figure 1 B is an example of a screen displayed on the display device 15 by the control unit 21 when performing preliminary imaging of sperm contained in semen provided by the sperm donor in the second preparation stage described above.

[0166] The screen displays an image column 61, a target number column 62, a shot number column 63, a shooting button 66, and an end button 67. In the image column 61, the microscope image shot by the camera 48 is displayed in real time. In the target number column 62, the number of targets to be shot is displayed. According to the optimal solution to the secretary problem, the number of targets to be shot is equivalent to about 37% of the number of sperms to be extracted. However, this number is usually an unrealistically large value, so it is set to an operational value. The shot number column 63 displays the number of sperms that have been shot.

[0167] When the user selects the capture button 66, the control unit 21 extracts the still image feature quantity and the moving image feature quantity of the image displayed in the image column 61. The control unit 21 inputs the extracted feature quantity into the process learning model 53 created for each stage to obtain the success probability. The control unit 21 determines whether the prediction result of each stage is normal based on a predetermined threshold. The control unit 21 creates a new record in the preliminary shooting DB 52 to record the prediction result.

[0168] The user determines whether to transition to the use of the target number column 62, the number of shots column 63, and the state of the sperm contained in the semen being observed. Figure 1 C. When the user selects the end button 67, the control unit 21 ends the preliminary imaging and transitions to the sperm selection stage. The control unit 21 may automatically transition to the sperm selection stage when the number of sperm that have undergone preliminary imaging reaches a target number.

[0169] Fig.11 Indicates that it is in use Figure 1C is an example of a screen displayed on the display device 15 by the sperm selection stage control unit 21. Displayed on the screen are an image field 61, a determination button 68, and an end button 67. In the image field 61, a microscope image captured by the camera 48 is displayed in real time.

[0170] When the user selects the determination button 68, the control unit 21 extracts the still image feature quantity and the moving image feature quantity of the image displayed in the image column 61. The control unit 21 inputs the feature quantity into the process learning model 53 to obtain the prediction success probability of each stage from fertilization to delivery. When the user selects the end button 67, the control unit 21 ends the processing.

[0171] Fig.12 The following is an example of a screen displayed by the control unit 21 on the display device 15 when the selection of the determination button 68 is accepted. The screen displays an image column 61, an evaluation column 65, and a next button 69. The evaluation column 65 includes a first evaluation column 651, a second evaluation column 652, a third evaluation column 653, a fourth evaluation column 654, and a comprehensive evaluation column 659.

[0172] In image column 61, use Fig.11 When the judgment button 68 is pressed on the screen described above, the microscope image displayed in the image field 61 is displayed in a static state. The user can check the candidate sperm being judged through the image field 61.

[0173] The control unit 21 displays the evaluation of the success probability in the first evaluation column 651, which is obtained by inputting the image feature quantity of the candidate sperm under determination into the process learning model 53 regarding the success or failure of fertilization. Figure 4 The deviation value performance within the second distribution h(X) is illustrated.

[0174] Similarly, the control unit 21 displays the evaluation of the success probability obtained by inputting the image feature quantity of the candidate sperm into the process learning model 53 regarding the success or failure of blastocyst formation in the second evaluation column 652. The control unit 21 displays the evaluation of the success probability obtained by inputting the image feature quantity of the candidate sperm into the process learning model 53 regarding the success or failure of implantation in the third evaluation column 653. The control unit 21 displays the evaluation of the success probability obtained by inputting the image feature quantity of the candidate sperm into the process learning model 53 regarding the success or failure of delivery in the fourth evaluation column 654.

[0175] The control unit 21 displays a comprehensive evaluation of the candidate sperm in the comprehensive evaluation column 659, which is a comprehensive evaluation of the evaluations displayed in the first evaluation column 651 to the fourth evaluation column 654. The comprehensive evaluation is a result of classifying the candidate sperm into "good", "normal", "bad", etc. based on, for example, the average value or the minimum value of the evaluations displayed in the first evaluation column 651 to the fourth evaluation column 654.

[0176] When the comprehensive evaluation is equal to or higher than a predetermined threshold, the control unit 21 may call the user's attention by any method such as sound generation.

[0177] The user determines whether to use the candidate sperm for microinsemination. If it is determined to be used, the user extracts the candidate sperm with a micropipette while confirming the candidate sperm through the eyepiece 43. If it is determined not to be used, or if the extraction is completed, the user selects the next button 69.

[0178] When the user selects the next button 69, the control unit 21 returns the screen displayed on the display device 15 to the user's Fig.11 The user can select a new sperm candidate.

[0179] The control unit 21 may also display a real-time microscope image in the image column 61. The user can simultaneously view the movement state of the candidate sperm and the evaluation column 65 to determine whether to use the candidate sperm. In addition, when the candidate sperm is about to move out of the microscope field of view, the user can appropriately operate the stage 42 to keep the candidate sperm in the microscope field of view.

[0180] Both a still image of the candidate sperm and a real-time microscope image may be displayed in the image field 61 by using screen division or sub-screen display.

[0181] Fig.13 is a flowchart showing the processing flow of the program used in the first preparation stage. Fig.13 Instructions for use Figure 1 A shows the processing flow of the first preparation stage. In the initial state, the sperm ID field, image data field, and process information field of the training data DB 51 record past microinsemination images and process information, and the sperm image feature field is blank.

[0182] The control unit 21 extracts a processing target record from the training data DB 51. The control unit 21 obtains an image from the image data field (step S501). The control unit 21 extracts the image feature value (step S502). Specifically, the control unit 21 uses, for example, Figure 6 As described above, a still picture is cut out from the image and the feature quantity of the moving picture is extracted. The control unit 21 inputs the cut out still picture into the image encoder 546 to obtain the feature quantity of the still picture.

[0183] The control unit 21 records the image feature in each subfield of the sperm image feature field of the processing target record (step S503). The control unit 21 determines whether the processing of the captured image recorded in the training data DB 51 has been completed (step S504). When it is determined that the processing has not been completed (No in step S504), the control unit 21 returns to step S501.

[0184] When it is determined that the processing is completed (yes in step S504), the control unit 21 uses the data recorded in the sperm image feature field and a subfield of the process information field as training data, performs supervised machine learning, and creates a process learning model 53 for a stage in the process from fertilization to delivery (step S511).

[0185] use Figure 7 To continue the description, when input data such as sperm image feature quantity is input to the input layer 531, the control unit 21 uses the back propagation algorithm or the like to calculate the parameters of the intermediate layer 532 so that the specified value is output to the output layer 533. When "OK" is recorded in the process subfield, the specified value is "1" in the "successful" neuron and "0" in the "unsuccessful" neuron. When "NG" is recorded in the process subfield, the specified value is "0" in the "successful" neuron and "1" in the "unsuccessful" neuron.

[0186] The control unit 21 stores the created process learning model 53 in the auxiliary storage device 23 (step S512). The control unit 21 obtains the number of "OK" and the number of "NG" for each of the process information subfields being processed (step S513).

[0187] By substituting the number of "OK" into p in formula (1) and the number of "NG" into q, we can obtain Figure 4 The control unit 21 stores the number of "OK" and the number of "NG" in the auxiliary storage device 23 in association with the subfield of the process information being processed (step S514).

[0188] The control unit 21 determines whether the processing of all subfields of the process information field has been completed, that is, determines whether the processing of all stages from fertilization to delivery has been completed (step S515).

[0189] If it is determined that the process is not completed (No in step S515), the control unit 21 returns to step S511. In addition, as described above, in step S511 after the second time, the control unit 21 may perform transfer learning using the created process learning model 53. If it is determined that the process is completed (Yes in step S515), the control unit 21 ends the process.

[0190] Fig.14 is a flow chart showing the process flow of the program used in the second preparation stage and sperm selection stage. Fig.14 Instructions for use Figure 1 B. Second stage of preparation and use of instructions Figure 1 C illustrates the processing flow during the sperm selection stage.

[0191] The control unit 21 starts the sampling distribution calculation subroutine (step S521). The sampling distribution calculation subroutine is a subroutine for performing preliminary imaging of sperm contained in the semen provided by the sperm donor and calculating the sampling distribution g(X). The processing flow of the sampling distribution calculation subroutine will be described later.

[0192] The control unit 21 obtains the information of each stage from fertilization to delivery. Fig.13 The control unit 21 calculates the number of OK and NG recorded in step S514 (step S522). The control unit 21 calculates the second distribution h(X) (step S523), which is obtained by substituting the first distribution f(X) and the sampling distribution g(X) into formula (3), wherein the first distribution f(X) is obtained by substituting the number of "OK" in p and the number of "NG" in q in formula (1), and the sampling distribution g(X) is calculated by the sampling distribution calculation subroutine.

[0193] The control unit 21 uses Fig.11 The screen for explanation is displayed on the display device 15. The user moves the stage 42 to search for sperm candidates for microinsemination. The user uses Fig.11 As described above, in a state where only one normal sperm is displayed in the image column 61, the determination button 68 is selected.

[0194] When the determination button 68 is selected, the control unit 21 obtains an image of sperm through the camera 48 (step S524). The control unit 21 extracts sperm image features (step S525). The control unit 21 inputs the sperm image features into the process learning model 53 for each stage from fertilization to delivery, and calculates the prediction success probability of normal growth at each stage (step S526).

[0195] The control unit 21 calculates an evaluation index for the prediction success probability calculated in step S526 within the second distribution h(X) calculated in step S523 (step S527). The evaluation index is, for example, a deviation value of the prediction success probability.

[0196] The control unit 21 determines the comprehensive evaluation of the candidate sperm based on the evaluation indexes for each stage from fertilization to delivery (step S528). For example, the control unit 21 determines whether the candidate sperm belongs to "good", "normal", or "bad" based on the average or minimum value of the deviation value of the predicted success probability at each stage. The control unit 21 displays the result of the operation on the display device 15. Fig.12 The description screen (step S529).

[0197] The control unit 21 determines whether the selection of the next button 69 is accepted (step S530). If it is determined that the selection of the next button 69 is accepted (yes in step S530), the control unit 21 displays the use of the next button 69 on the display device 15. Fig.11 The description screen is displayed, and returns to step S524.

[0198] When it is determined that the selection of the next button 69 has not been accepted even after the predetermined time has elapsed, or when it is determined that an end instruction has been accepted (No in step S530 ), the control unit 21 ends the processing.

[0199] Fig.15 1 is a flowchart showing the processing flow of the sampling distribution calculation subroutine. The sampling distribution calculation subroutine is a subroutine that performs preliminary imaging of sperm contained in semen provided by a sperm donor and calculates a sampling distribution g(X).

[0200] The control unit 21 uses Fig.10 The screen for explaining is displayed on the display device 15. In addition, the control unit 21 uses Fig.11 The screen of the explanation is displayed on the display device 15. The user moves the stage 42 to find normal sperm that can be used for microinsemination. Fig.10 As described above, the capture button 66 is selected in a state where only one normal sperm is displayed in the image column 61 .

[0201] When the capture button 66 is selected, the control unit 21 acquires an image of sperm through the camera 48 (step S541). The control unit 21 extracts sperm image features (step S542). The control unit 21 creates a new record in the preliminary capture DB 52 (step S543).

[0202] The control unit 21 inputs the sperm image feature value into the process learning model 53 of one of the stages from fertilization to delivery, and obtains the prediction success probability (step S544). The control unit 21 determines whether the prediction success probability is greater than a predetermined threshold (step S545).

[0203] If it is determined that the value is greater than the threshold value (Yes in step S545), the control unit 21 records "OK" in the field corresponding to the stage of processing in the record created in step S543 (step S546). If it is determined that the value is less than the threshold value (No in step S545), the control unit 21 records "NG" in the field corresponding to the stage of processing in the record created in step S543 (step S547).

[0204] The control unit 21 determines whether the processing of all stages from fertilization to delivery has been completed (step S548). When it is determined that it has not been completed (No in step S548), the control unit 21 returns to step S544.

[0205] If it is determined that the preparatory shooting has been completed (Yes in step S548), the control unit 21 determines whether to end the preliminary shooting (step S549). Fig.10 When the selection of the end button 67 is accepted on the screen described above, the control unit 21 determines that the preliminary imaging is to be ended.

[0206] If it is determined that the preliminary shooting is not to be completed (No in step S549), the control unit 21 returns to step S541. If it is determined that the preliminary shooting is to be completed (Yes in step S549), the control unit 21 obtains the number of "OK" and the number of "NG" for the subfield being processed in the prediction result field (step S550).

[0207] By substituting the number of "OK" for a in formula (2) and the number of "NG" for b, we can get Figure 4 The control unit 21 records the number of "OK" and the number of "NG" in the auxiliary storage device 23 in association with the prediction result subfield being processed (step S551).

[0208] The control unit 21 determines whether the processing of all subfields of the prediction result field has been completed, that is, whether the processing of all stages from fertilization to delivery has been completed (step S552). If it is determined that it has not been completed (No in step S552), the control unit 21 returns to step S550. If it is determined that it has been completed (Yes in step S552), the control unit 21 ends the processing.

[0209] According to the present embodiment, it is possible to provide the information processing system 10 that supports the selection of sperm with a high probability of success in microinsemination.

[0210] According to this embodiment, by using Bayesian inference, it is possible to accurately predict the posterior probability distribution of the probability (probability of success) of normal growth of the embryo after microinsemination, such as whether the sperm contained in the semen is fertilized successfully, whether it grows normally to each stage such as a 4-cell embryo, a morula, and a blastocyst, whether the pregnancy is successful, whether there are abnormalities in the pregnancy test, and whether the delivery is successful. Among them, the above-mentioned Bayesian inference uses the first distribution f(X) as the prior probability distribution, and the success probability predicted by sampling the sperm in the semen is used as the likelihood distribution. The above-mentioned first distribution f(X) is based on data recording the process of microinsemination using IMSI performed in the past.

[0211] According to this embodiment, since the evaluation of each sperm within the distribution of various success probabilities related to sperm contained in semen is displayed in real time, it is possible to support the insemination technician in determining whether to use the sperm under observation for microinsemination. Therefore, even a less experienced insemination technician can correctly select sperm just like an experienced insemination technician.

[0212] According to this embodiment, since the success probability is evaluated at each stage such as the fertilization stage and the blastocyst formation stage, the incubator can select sperm in consideration of the clinical characteristics and condition of the patient. For example, when there are many eggs that can be extracted, even if the success probability of fertilization is low, the incubator can select sperm with a high success probability after implantation.

[0213] By extracting the feature amount of the captured image using the image encoder 546, the input dimension of the process learning model 53 can be reduced. Thus, it is possible to provide an information processing system 10 that evaluates sperm in real time. Since the captured image can be classified with high accuracy by the image encoder 546, it is possible to provide an information processing system 10 that evaluates sperm with high accuracy.

[0214] The image encoder 546 may be integrated with the process learning model 53. The integrated process learning model 53 is generated by connecting the output of the image encoder 546 and the input layer 531 of the process learning model 53.

[0215] By integrating with the image encoder 546, it is possible to realize the process learning model 53 that outputs the success probability of each stage in the process from fertilization to delivery when the image of sperm is input. In addition, when the image encoder is integrated with the process learning model 53, it is preferable to use Figure 8 In the training data DB 51 described above, a field for recording a sperm image extracted from a moving image file is provided instead of a subfield for recording a still image feature amount.

[0216] The image encoder 546 and the CNN for determination used when extracting a still image containing the entire sperm from a moving image file may also be integrated with the process learning model 53. In this case, when using Figure 8 In the training data DB 51 described above, there is no need to record the subfield for recording the still image feature amount.

[0217] It is preferable to add additional data to the training data, which is data that records the image feature quantity of the image captured in the sperm selection stage in association with the subsequent process. By relearning using new training data, the accuracy of the process learning model 53 can be improved. The process learning model 53 updated by relearning can also be sent to other microinsemination implementation institutions through the network.

[0218] For example, for items such as the health status of newborns, a score-based evaluation of 0% to 100% can be performed instead of a binary evaluation of success or failure. The first distribution f(X) of the items for which the score-based evaluation is performed is the probability density distribution of the scores of all recorded cases approximated by the Beta distribution. p and q are calculated using the mean E and variance V of the frequency distribution of the scores. The first distribution f(X) in this case is represented by equation (4).

[0219]

Number 4

[0220] Y=f(X)=C4X p-1 (1-X) q-1 ……(4)

[0221]

[0222] E represents the mean of the scores.

[0223] V represents the variance of the scores.

[0224] C4 represents a normalization constant.

[0225] The first distribution f(X) can also be calculated by, for example, taking any distribution such as uniform distribution as the prior probability distribution and performing Bayesian update on each piece of training data. The sampling distribution g(X) can also be calculated by, for example, taking any distribution such as the same distribution as the uniform distribution or the first distribution f(X) as the prior probability distribution and performing Bayesian update on each spermatozoon subjected to preliminary imaging.

[0226] The sampling distribution g(X) may also use the Beta distribution. Specifically, the equation (5) representing the Beta distribution is used instead of the above equation (2).

[0227]

Number 5

[0228] Y=g(X)=C5X a-1 (1-X)b-1 ……(5)

[0229] a represents the number of preliminary captured images estimated to be normal growth at each growth stage.

[0230] b represents the number of preliminary captured images estimated to be unable to grow normally to each growth stage.

[0231] C5 represents a normalization constant.

[0232] Gaussian distribution may be used for both or either of the first distribution f(X) and the sampling distribution g(X).

[0233] If used Figure 7 As described above, information related to the sperm donor and the egg donor, such as the age, health status, past infertility treatment history, or genomic information such as gene mutation, etc., of the sperm donor and the egg donor can also be added to the input layer 531 of the process learning model 53. By doing so, it is possible to realize a process learning model 53 that predicts the probability of success based on the influence of age increase, etc.

[0234] When taking a still picture of sperm, the amount of illumination light emitted from the illumination unit 44 can be increased and the picture can be taken with a short exposure time. Even sperm that is actively moving can be taken with a clear still picture. By using a clear still picture, a process learning model 53 that can clearly distinguish sperm with a high probability of success from sperm with a low probability of success can be created, and candidate sperm can be evaluated with high accuracy.

[0235] The camera 48 preferably has a high resolution. By photographing sperm with a high resolution, clear moving images and still images can be captured. Therefore, a process learning model 53 that can clearly distinguish sperm with a high probability of success from sperm with a low probability of success can be created, and candidate sperm can be evaluated with high accuracy. In order to obtain practical accuracy, it is preferred to photograph one sperm with a resolution of at least 32 pixels × 32 pixels or more.

[0236] The camera 48 is preferably capable of capturing high-contrast images. By using high-contrast images, individual differences in sperm can be clearly identified. Therefore, a process learning model 53 that can clearly identify sperm with a high probability of success and sperm with a low probability of success can be created, and candidate sperm can be evaluated with high accuracy.

[0237] [Implementation Method 2]

[0238] The present embodiment relates to an information processing system 10 that calculates a sampling distribution g(X) based on the success probability obtained by a process learning model 53. Description of parts common to the first embodiment will be omitted.

[0239] Fig.161 is an explanatory diagram for explaining the record structure of the preliminary shooting DB 52 according to the second embodiment. Figure 1 The second preparation stage described in B is a DB that records the prediction results using the process learning model 53. The preparation shooting DB 52 has a patient ID field, a shooting date field, and a prediction result field. The prediction result field has a fertilization field, a blastocyst formation field, an implantation field, a delivery field, and a health status field. The preparation shooting DB 52 has one field for one preparation shooting.

[0240] The patient ID field records the unique patient ID assigned to the sperm donor. The shooting date field records the shooting date of the image. Each subfield of the prediction result field records the predicted value of the probability of normal arrival at each stage in percentage units.

[0241] The control unit 21 extracts sperm image features from the images captured by the camera 48 and inputs them into the process learning model 53 created for each stage to obtain the success probability. The control unit 21 records the obtained probability in each field of the preliminary imaging DB 52.

[0242] In this embodiment, these success probabilities are approximated by the beta distribution g(X) represented by equation (6). When determining the variables p and q in the approximation equation, the mean and variance are calculated from the frequency distribution of the success probability and substituted into E and V respectively.

[0243]

Number 6

[0244] Y=g(X)=C6X p-1 (1-X) q-1 ……(6)

[0245]

[0246] E represents the mean (expected value) of the distribution of the probability of successful prediction of normal growth to each stage.

[0247] V represents the variance of the distribution of the predicted success probability of normal growth to each stage.

[0248] C6 represents a normalization constant.

[0249] Fig.17 This is a flowchart showing the processing flow of the sampling distribution calculation subroutine in the second embodiment. Fig.17 The sampling distribution calculation routine shown is used instead of Fig.15 The processing flow before step S544 and the use of the sampling distribution calculation subroutine are as follows: Fig.15 The processing of the sampling distribution calculation subroutine described above is the same, so the description is omitted.

[0250] The control unit 21 records the predicted success probability obtained in step S544 in the field corresponding to the stage being processed in the record created in step S543 (step S561). The control unit 21 determines whether the processing of all stages from fertilization to delivery has been completed (step S562). If it is determined that it has not been completed (No in step S562), the control unit 21 returns to step S544.

[0251] If it is determined that the preparatory shooting has been completed (Yes in step S562), the control unit 21 determines whether to complete the preparatory shooting (step S563). If it is determined that the preparatory shooting has not been completed (No in step S563), the control unit 21 returns to step S541.

[0252] When it is determined that the preliminary shooting is to be ended (Yes in step S563), the control unit 21 calculates the mean value E and the variance V for one subfield of the prediction result field of the preliminary shooting DB 52. The control unit 21 calculates the variables p, q, and C6 according to equation (6) (step S564). The control unit 21 records the variables p, q, and C6 in the auxiliary storage device 23 in association with the prediction result subfield being processed (step S565).

[0253] The control unit 21 determines whether the processing of all subfields of the prediction result field has been completed, that is, whether the processing of all stages from fertilization to delivery has been completed (step S566). If it is determined that it has not been completed (No in step S566), the control unit 21 returns to step S564. If it is determined that it has been completed (Yes in step S566), the control unit 21 ends the processing.

[0254] According to the present embodiment, it is not necessary to determine a threshold value for judging "OK" and "NG" when preparing the preliminary imaging DB 52. Therefore, it is possible to provide an information processing system 10 for evaluating sperm candidates without being affected by the threshold setting.

[0255] According to the present embodiment, it is possible to provide an information processing system 10 in which, when there are many sperms with a prediction success probability of around 50%, the number of preliminary shots is small and the sampling distribution g(X) is stable.

[0256] [Implementation method 3]

[0257] This embodiment relates to an information processing system 10 that displays an evaluation index of a predicted success probability of a candidate sperm and displays the reliability of the evaluation index of the predicted success probability. Description of parts common to the first embodiment will be omitted.

[0258] Fig.18 This is an explanatory diagram showing a screen displayed by the information processing device 20 according to the third embodiment. Fig.18 The picture shown is the sperm selection stage replaced Fig.12The screen shown in FIG. 1 is an example of a screen displayed on the display device 15. The screen displays an image field 61, an evaluation field 65, and a next button 69. The evaluation field 65 includes a first evaluation field 651, a second evaluation field 652, a third evaluation field 653, and a fourth evaluation field 654.

[0259] The first evaluation column 651 to the fourth evaluation column 654 respectively display the deviation value and the reliability of the evaluation index. The reliability is, for example, the inverse of the random uncertainty of data inhomogeneity. As the random uncertainty conforming to the Gaussian distribution, the variance σ 2 =Σ(X-Xm) 2 / N (N is the number of samples, Xm is the average value of X) is the reciprocal of the accuracy β = 1 / σ 2 For reliability.

[0260] Reliability is the inverse of the uncertainty of the model or parameter uncertainty of supervised machine learning, or the inverse of the uncertainty of identification such as the uncertainty of integrity. The uncertainty of identification can be calculated by using Monte Carlo Dropout Sampling, which replaces the parameters of the machine learning model with Bernoulli distribution. In addition, by adding appropriate constants to the calculated reliability and multiplying it by other appropriate constants, it can be displayed as a reliability, for example, converted to a value from 0 to 100, so that the trainer can intuitively understand the level of reliability.

[0261] The culture technician can decide on the sperm to be used for microinsemination by considering both the variance value and the reliability, for example, avoiding selecting a candidate sperm if the candidate sperm has a high variance value but a low reliability.

[0262] [Implementation Method 4]

[0263] This embodiment relates to an information processing system 10 for automatically judging sperm candidates. The description of the parts common to the first embodiment will be omitted.

[0264] Fig.19 1 is an explanatory diagram for explaining the structure of the information processing system 10 according to Embodiment 4. The information processing device 20 has a stage I / F 26. The stage I / F 26 is, for example, a USB terminal. The microscope 41 has a stage moving unit 46 for moving the stage 42. The stage moving unit 46 is connected to the stage I / F 26.

[0265] The control unit 21 controls the stage moving unit 46 via the stage I / F 26. The control unit 21 controls the stage moving unit 46 so that, for example, the field of view of the microscope 41 scans the entire observation container 421. The control unit 21 may control the stage moving unit 46 according to a user's instruction.

[0266] Fig. 20This is an explanatory diagram for explaining the record structure of the normal sperm determination DB. The normal sperm determination DB is training data used in supervised machine learning for distinguishing normal sperm and other things. The normal sperm determination DB has an image field and a determination field. The image field records images taken of normal sperm, abnormal sperm, or foreign matter, etc. The determination field records the result of the culture technician's determination of whether the image is "normal sperm", "abnormal sperm", or "foreign matter". The normal sperm determination DB has one record for one image.

[0267] Fig.21 57 is an explanatory diagram showing the structure of the normal sperm determination model 57. The normal sperm determination model 57 is a neural network including an input layer 571, an intermediate layer 572, and an output layer 573. Fig.21 , the case where the normal sperm determination model 57 is a CNN is illustrated.

[0268] The input layer 571 is input with the image recorded in the image field of the normal sperm determination DB. Specifically, the input layer 571 has the same number of neurons as the number of image pixels, and each neuron is input with the pixel value of each pixel of the image. The output layer 573 has a total of 3 neurons, which respectively output the probability of being determined as "normal sperm", the probability of being determined as "abnormal sperm", and the probability of being determined as "foreign matter".

[0269] The control unit 21 performs supervised machine learning to calculate the parameters of the intermediate layer 572 using a back propagation algorithm or the like, and outputs the determination result recorded in the determination field of the normal sperm determination DB from the output layer 543. For example, when "normal sperm" is recorded in the determination field, the control unit 21 calculates the parameters of the intermediate layer 572 so that the probability of "normal sperm" is 1, and the probability of "abnormal sperm" and the probability of "foreign matter" are 0.

[0270] By inputting the image captured by the camera 48 to the learned normal sperm determination model 57, the control unit 21 can determine whether normal sperm, abnormal sperm, or foreign matter is captured.

[0271] The normal sperm determination model 57 can also be created using transfer learning, which is to add learning to the normal sperm determination DB using existing open source models such as VGG (Visual Geometry Group)-16, ResNET (Residual Network)-50, or Xception. By using the open source model, it is possible to create an accurate normal sperm determination model 57 using less training data.

[0272] Figure 22 to Figure 24 This is an explanatory diagram showing a screen displayed by the information processing device 20 according to the fourth embodiment. Figure 22 to Figure 24 The picture shown is the sperm selection stage replaced Fig.12 The screen shown is an example of a screen displayed on the display device 15. An image field 61 and an evaluation field 65 are displayed on the screen.

[0273] The image field 61 can display in real time the image captured by the camera 48. The control unit 21 uses a known image recognition technology to determine whether the evaluation object is included in the field of view. The control unit 21 can also input the image into a pre-prepared CNN for determination to determine whether the evaluation object is included in the field of view.

[0274] The evaluation column 65 can display the evaluation result of the image displayed in the image column 61 in real time. Fig. 22 In the example shown, the field of view contains normal spermatozoa, showing Fig.12 Same evaluation results.

[0275] Fig.23 In the example shown, abnormal sperm is included in the visual field, and "abnormal sperm" is displayed in the comprehensive evaluation column 659. For abnormal sperm, the process learning model 53 is not used to obtain the prediction success probability, and "-" is displayed in the first evaluation column 651 to the fourth evaluation column 654.

[0276] Fig.24 In the example shown, the field of view does not include the evaluation object, and "cannot be determined" is displayed in the comprehensive evaluation column 659. Since there is no evaluation object, the process learning model 53 is not used to obtain the prediction success probability, and "-" is displayed in the first evaluation column 651 to the fourth evaluation column 654.

[0277] Fig.25 4 is a flowchart showing the processing flow of the program of Embodiment 4. In this embodiment, the stage 42 moves forward, backward, left, and right automatically or in accordance with the user's instruction to scan the microscope field.

[0278] The control unit 21 starts the sampling distribution calculation subroutine (step S601). The sampling distribution calculation subroutine is used to perform Fig.15 or Fig.17 This is a subroutine that calculates the sampling distribution g(X) by performing the same processing as the sampling distribution calculation subroutine described above.

[0279] In the subroutine started in step S601, instead of using Fig.15 or Fig.17 In step S541 of the subroutine described above, the control unit 21 starts a sperm image acquisition subroutine. The sperm image acquisition subroutine is a subroutine for automatically acquiring an image of normal sperm from the microscope 41. The processing flow of the sperm image acquisition subroutine will be described later.

[0280] The control unit 21 obtains the information of each stage from fertilization to delivery. Fig.13The control unit 21 calculates the second distribution h(X) based on equations (1) to (3) (step S603).

[0281] The control unit 21 uses a known image analysis technique to detect that there is an object in the field of view captured by the camera 48 (step S604). Fig.21 The normal sperm determination model 57 described above receives an image including the object to be photographed as input, and obtains a determination result (step S605).

[0282] The control unit 21 determines whether the probability of the photographed object being normal sperm is above a predetermined threshold value (step S606). If it is determined to be above the threshold value (yes in step S606), the control unit 21 extracts the sperm image feature value (step S525). The subsequent processing until step S528 is the same as that using Fig.14 The processing of the described program is the same, so the description is omitted. The control unit 21 displays the use of the display device 15 Fig. 22 The description screen (step S607).

[0283] If it is determined that the probability of the object being photographed is less than the threshold value (No in step S606), the control unit 21 determines whether the probability of the object being photographed being abnormal sperm is greater than a predetermined threshold value (step S611). If it is determined that the probability of the object being photographed is greater than the threshold value (Yes in step S611), the control unit 21 displays the image on the display device 15. Fig.23 If the value is less than the threshold value (No in step S611), the control unit 21 displays the abnormal sperm on the display device 15. Fig.24 A screen indicating that the determination cannot be made is displayed as described above (step S613).

[0284] After step S607, step S612 or step S613, the control unit 21 determines whether the selection of the next button 69 is accepted (step S614). If it is determined that the selection of the next button 69 is accepted (yes in step S614), the control unit 21 returns to step S604.

[0285] When it is determined that the selection of the next button 69 has not been accepted even after the predetermined time has elapsed, or when it is determined that the end instruction has been accepted (No in step S514), the control unit 21 ends the processing.

[0286] Fig.26 4 is a flowchart showing the processing flow of the sperm image acquisition subroutine. The sperm image acquisition subroutine is a subroutine for automatically acquiring an image of normal sperm from the microscope 41. The sperm image acquisition subroutine replaces the use of Fig.15 and Fig.17 The sampling distribution calculation subroutine step S541 is used as described above.

[0287] The control unit 21 detects the presence of an object in the field of view using a known image analysis technique (step S571). The control unit 21 inputs the image containing the object into the normal sperm determination model 57 to obtain the probability that the object is a normal sperm (step S572).

[0288] The control unit 21 determines whether the probability of the photographed object being normal sperm is above a predetermined threshold value (step S573). If it is determined to be less than the threshold value (No in step S573), the control unit 21 returns to step S571. If it is determined to be above the threshold value (Yes in step S573), the control unit 21 ends the processing.

[0289] According to this embodiment, it is possible to provide an information processing system 10 that automatically extracts and determines normal sperm. For example, it is possible to provide an information processing system 10 that does not increase the burden on the culture technician even when there are very few normal sperm.

[0290] By separating the normal sperm determination model 57 for extracting normal sperm and the process learning model 53 for evaluating normal sperm, it is possible to provide an information processing system 10 that performs high-precision determination.

[0291] [Implementation method 5]

[0292] This embodiment relates to an information processing system 10 that displays the evaluation results of candidate sperm using a graph. The description of the parts common to the third embodiment will be omitted.

[0293] Fig. 27 This is an explanatory diagram showing a screen displayed by the information processing device 20 according to the fifth embodiment. Fig. 27 The picture shown is the sperm selection stage replaced Fig.18 The screen shown is an example of a screen displayed on the display device 15. The screen displays an image field 61, an evaluation field 65, and a patient information field 64. The evaluation field 65 includes a first evaluation field 651, a second evaluation field 652, a third evaluation field 653, a fourth evaluation field 654, and a fifth evaluation field 655.

[0294] The age, past and current medical history, and infertility treatment history of the sperm donor and the egg donor are displayed in the patient information column 64. In the patient information column, "M" represents the sperm donor, and "F" represents the egg donor.

[0295] The candidate sperm being evaluated is displayed as a still image in the image field 61. The image field 61 may also display a microscope image captured by the camera 48 in real time.

[0296] The first evaluation column 651 to the fourth evaluation column 654 show graphs of the second distribution Y=h(X) at each stage. The vertical axis of the graph represents the success probability as the probability variable X, and the horizontal axis represents the probability Y corresponding to the probability variable X. In each graph, the success probability of the candidate sperm is shown by the indicator line 81.

[0297] Below each graph, the success rate calculated for the candidate sperm, the deviation value of the success rate in the second distribution, and the reliability are displayed. When the success rate of the candidate sperm is greater than the maximum success rate of sperm in the same semen that has been evaluated in the past, the word "Best" is displayed below the reliability.

[0298] A graph showing the distribution of sperm abnormality is displayed in the fifth evaluation column 655. The vertical axis of the graph shows the abnormality of sperm as the probability variable X, and the horizontal axis shows the probability Y corresponding to the probability variable X. In the graph, the abnormality of the candidate sperm is displayed by the index line 81.

[0299] Below the graph, the abnormality calculated for the candidate sperm, the deviation value of the abnormality in the abnormality distribution, and the reliability are displayed. If the abnormality of the candidate sperm is below the minimum abnormality of sperm in the same semen that has been evaluated in the past, the word "Best" is displayed below the reliability.

[0300] The abnormality of spermatozoa is determined by using Fig.21 The probability of sperm being abnormal sperm obtained by the normal sperm determination model 57 described above. The abnormality of sperm can also be calculated based on the similarity between the shape of sperm set as "normal" and the shape of candidate sperm. The curve graph in the fifth evaluation column 655 is, for example, the distribution of sperm abnormality of men of the same age as the sperm donor.

[0301] As the graph of the fifth evaluation column 655, a second distribution may be used, which is calculated by using the sperm abnormality distribution of men of the same age group as the first distribution and the abnormality distribution of sperm captured by the preliminary shooting as the sampling distribution. In this case, the preliminary shooting is first automatically performed in the same manner as in Embodiment 4 to create a sampling distribution of the abnormality of all sperm contained in the semen. Then, for the candidate sperm determined to be of normal shape, a sampling distribution of each stage from fertilization to delivery is created.

[0302] According to this embodiment, the trainer can intuitively grasp the position of the candidate sperm in the distribution based on the index line 81. Therefore, the trainer can quickly determine whether to use the candidate sperm for microinsemination.

[0303] Alternatively, a moving image file obtained by shooting the candidate sperm for a few seconds may be temporarily stored in the auxiliary storage device 23 and repeatedly played and displayed on the image column 61. The culture technician can fully observe the activity of the candidate sperm through the moving image and determine whether to use it for microinsemination. Buttons may also be displayed around or inside the image column 61 to instruct fast forward, fast rewind, pause, slow motion or fast motion of the repeatedly played moving image.

[0304] The control unit 21 may receive a designation of a frame from which to extract still image feature quantities during repeated playback of a moving image file and re-evaluate the sperm candidates. The technician can evaluate the sperm candidates based on a desired frame.

[0305] The moving image files shot in the past may also be displayed in the image column 61. For example, it is possible to provide an information processing system 10 used for seminars, teacher training, information exchange with other facilities, etc. in a medical facility. Using the moving image files shot in the past, it is also possible to confirm the operation after updating the process learning model 53 or the image encoder 546.

[0306] The camera 48 may also be capable of high-speed photography. The moving image file of the candidate sperm photographed by high-speed photography is displayed in slow motion in the image column 61, thereby allowing the incubator to observe the activity of the candidate sperm. By doing so, the incubator can correctly confirm the natural activity of the sperm and the defocus of the microscope image without using a high-viscosity solution. The incubator can obtain an image of the candidate sperm suitable for evaluation by adjusting the position of the objective lens, etc. Further, it is possible to avoid injecting a high-viscosity solution into the egg together with the sperm during microinsemination.

[0307] [Implementation Method 6]

[0308] Fig.28 1 is a functional block diagram of the information processing device 20 according to Embodiment 6. The information processing device 20 includes a captured image acquisition unit 71, an input unit 72, and an output unit 73. The captured image acquisition unit 71 acquires a captured image in which a candidate sperm used in microinsemination is captured.

[0309] The input unit 72 receives an image of sperm and inputs the acquired image into the learning model 53, which outputs a prediction of whether microinsemination using the sperm will be successful. The output unit 73 outputs the prediction output from the learning model 53 based on the input image.

[0310] The learning model 53 has an input layer 571, an image encoder 546, an intermediate layer 572, and an output layer 573. The input layer 571 receives an image of sperm, and the output layer 573 outputs the predicted success probability of whether the sperm has grown normally to each stage after microinsemination using sperm.

[0311] The parameters of the intermediate layer 572 are learned using training data in which images of sperm used in microinsemination in the past are recorded in association with images of sperm and whether or not the sperm has grown normally to each stage after microinsemination using the sperm.

[0312] The learning model 53 enables the computer to implement the following functions: when an image of a candidate sperm to be used for micro-insemination is input to the input layer 571, after calculations in the intermediate layer 572, a prediction regarding whether the candidate sperm will grow normally to each stage when micro-insemination is performed is output from the output layer 573.

[0313] The learning model 53 further includes an image encoder 546 provided between the input layer 571 and the intermediate layer 572 , and extracting image feature quantities from the captured image. The image feature quantities extracted by the image encoder 546 are output to the intermediate layer 572 .

[0314] [Implementation method 7]

[0315] In the present embodiment, the information processing system 10 of the present embodiment is realized by operating a general-purpose computer 90 and a program 97 in combination. Fig.29 This is an explanatory diagram showing the configuration of an information processing system 10 according to Embodiment 7. Description of parts common to Embodiment 1 will be omitted.

[0316] The information processing system 10 of the present embodiment includes a computer 90 and a microscope 41 .

[0317] The computer 90 includes a control unit 21, a main storage device 22, an auxiliary storage device 23, a communication unit 24, a display I / F 25, an imaging I / F 28, a reading unit 29, and a bus. The computer 90 is an information device such as a general-purpose personal computer, a tablet computer, or a server computer.

[0318] The program 97 is recorded in a portable recording medium 96. The control unit 21 reads the program 97 through the reading unit 29 and stores it in the auxiliary storage device 23. The control unit 21 can also read the program 97 stored in a semiconductor memory 98 such as a flash memory installed in the computer 90. The control unit 21 can also download the program 97 from another server computer (not shown) connected through the communication unit 24 and a network (not shown) and store it in the auxiliary storage device 23.

[0319] The program 97 is installed as a control program of the computer 90, loaded into the main storage device 22, and executed. Thus, the computer 90 realizes the functions of the above-mentioned information processing device 20.

[0320] [Embodiment 8]

[0321] This embodiment relates to an information processing system 10 for determining whether a sperm under observation is an X sperm having an X chromosome or a Y sperm having a Y chromosome. Descriptions of parts common to Embodiment 1 are omitted. In the following description, the influence of sex chromosome mutation is not considered.

[0322] For example, various sex-linked genetic diseases such as red-green color blindness, hemophilia, and Alport syndrome are known. Sex-linked genetic diseases are genetic diseases that originate from abnormal sex chromosomes. The incidence of sex-linked genetic diseases in children born to parents with sex-linked genetic diseases, and the carrier rate of genes that pass on sex-linked genetic diseases to their offspring even though the children themselves do not develop the disease, vary depending on the type of sex-linked genetic disease and the sex of the child.

[0323] Fig.30 This is a table showing the incidence and carrier rate of sex-linked inferior (recessive) genetic diseases (in the following description, "inferior" and "recessive" are used to have the same meaning). Sex-linked inferior genetic diseases are diseases that are inherited through the X chromosome and will not develop as long as there is a normal X chromosome. Males have one X chromosome, so as long as there is an abnormality, the disease will develop. If only one of the two X chromosomes that women have is abnormal, the disease will not develop, but if both are abnormal, the disease will develop. For example, sex-linked inferior genetic diseases such as red-green color blindness and hemophilia are known.

[0324] Fig.30 In the left column, the column indicates whether the sex chromosome of the sperm donor is normal or abnormal. In the abnormal case, the sperm donor's X chromosome is abnormal and will suffer from disease. Fig.30 The second column from the left of is divided into the case where X sperm is used and the case where Y sperm is used. When X sperm is used, the gender of the child is female. When Y sperm is used, the gender of the child is male. In the following description, the difference between whether the sperm is X sperm or Y sperm is sometimes recorded as the gender of the sperm.

[0325] Fig.30 In the diagram, the three columns on the right indicate whether the sex chromosomes of the egg donor are normal or abnormal. In the case of abnormality, there are two forms, one is heterozygous, where one X chromosome is abnormal and the other is normal, and the other is homozygous, where both X chromosomes are abnormal. In the case of heterozygous, the patient becomes a carrier of the gene without developing the disease. In the case of homozygous, the disease occurs.

[0326] The incidence rate indicates the probability of a child suffering from a sex-linked genetic disease. The carrier rate indicates the probability that a child's sex chromosome has the genetic gene for a sex-linked genetic disease, that is, the probability that the child's genes are abnormal.

[0327] For example, when the sex chromosome of the egg donor is normal and the sex chromosome of the sperm donor is abnormal, the incidence rate is 0% regardless of the gender of the child, the carrier rate of girls is 100% and the carrier rate of boys is 0%. Fig.30 In the area enclosed by the rounded boxes, the incidence or carrier rate varies depending on the child's gender.

[0328] Fig.31 This is a table showing the incidence and carrier rate of sex-linked dominant (dominant) genetic diseases (in the following description, "dominant" and "dominant" are used for the same meaning). Sex-linked dominant genetic diseases are diseases that are inherited through the X chromosome and will develop as long as there is an abnormal X chromosome. Men have one X chromosome, so as long as there is an abnormality, they will develop the disease. Women have two X chromosomes, and if one or both have an abnormality, they will develop the disease. For example, sex-linked dominant genetic diseases such as Rett syndrome and Alport syndrome are known.

[0329] Fig.31 The structure of the table shown is similar to Fig.30 The tables described in are the same, so the description is omitted. Fig.31 In the figure, also in the area enclosed by the rounded box, the incidence or carrier rate varies depending on the child's gender.

[0330] Fig.32 This is a table showing the incidence and carrier rate of sex-linked genetic diseases inherited through the Y chromosome. Women do not have a Y chromosome, so the genes associated with such diseases must be normal. It can be considered that part of male infertility includes diseases inherited through the Y chromosome, such as Y chromosome microdeletions.

[0331] Fig.32 The left end of Fig.30 Similarly, it indicates whether the sex chromosomes of the sperm donor are normal or abnormal. The egg donor must be normal. Fig.32 In the area enclosed by the rounded boxes, the incidence or carrier rate varies depending on the child's gender.

[0332] In the case of sex chromosome abnormalities in the sperm donor and the egg donor, the gender of the child can be selected by determining the gender of the sperm, which can reduce the incidence and carrier rate of the child after birth.

[0333] It is known that due to disease, embryos with abnormal sex chromosomes have a low probability of success from implantation to birth and are prone to miscarriage. By determining the sex of sperm and selecting the sex of the child, the success rate of microinsemination can be increased.

[0334] It is known that in human genes, the genome amount of X sperm is about 3% more than that of Y sperm. It is also known that along with the difference in genome amount, the head length and circumference length of X sperm are about 5% larger than the corresponding parts of Y sperm, and the mass of X sperm is about 2.8% heavier than that of Y sperm. Since X sperm is heavier, Y sperm moves faster.

[0335] Thus, there is a difference in appearance between X sperm and Y sperm. However, it is difficult for a culture technician to identify and extract the desired sperm in an instant. In this embodiment, a sex determination learning model 56 (see FIG. 5 ) generated by machine learning is provided. Fig.34 ) An information processing system 10 for determining whether a sperm under observation is an X sperm or a Y sperm.

[0336] Fig.33 : is an explanatory diagram for explaining the record structure of the gender training data DB58. The gender training data DB58 is a DB that records the image of the sperm used in microinsemination and the gender of the child born by microinsemination in association with each other. The gender training data DB58 has a sperm ID field, an image data field, a sperm image feature field, and a gender field. The sperm ID field, the image data field, and the sperm image feature field are associated with the sperm used in microinsemination. Figure 8 Since the fields of the training data DB 51 described above are the same, the description thereof will be omitted.

[0337] The sex field records the sex of the child born by microinsemination. The sex field may also record the sex of the embryo or fetus determined by prenatal diagnosis such as NIPT (Non-Invasive Prenatal Genetic Testing). The sex training data DB 58 has one record for one sperm used in microinsemination.

[0338] Fig.34 56 is an explanatory diagram for explaining the gender determination learning model 56. The gender determination learning model 56 is a neural network including an input layer 561, an intermediate layer 562, and an output layer 563. Figure 7 , the case where the gender determination learning model 56 is a CNN is illustrated. In addition, the convolution layer and the pooling layer are omitted in the illustration.

[0339] When the feature values ​​of still images and moving images are input to the input layer 561, the gender determination learning model 56 outputs the probability of X sperm and the probability of Y sperm to the output layer 563. The information input to the input layer 561 is the same as the information input to the user. Fig.34 The input of the process learning model 53 described is the same, so the description is omitted.

[0340] When the input layer 561 is input with feature quantities of still images and moving images, the gender determination learning model 56 outputs the probability that the sperm under observation is an X sperm and the probability that it is a Y sperm to the output layer 563. In the learning stage, the control unit 21 performs supervised machine learning by using the gender training data DB58 obtained using the back propagation algorithm and calculating the parameters of the intermediate layer 562. Fig.33 Explanatory.

[0341] Supervised machine learning can be performed using any method such as logistic regression, SVM, random forest, CNN, RNN, or XGBoost.

[0342] The sex determination learning model 56 may be created using any computer. The created sex determination learning model 56 is sent to the information processing device 20 used by the microinsemination implementation institution via a network or the like, and is recorded in the auxiliary storage device 23 .

[0343] The gender determination learning model 56 can also be used with Figure 7 In this case, the output layer 563 of the gender determination learning model 56 has a success node that outputs the probability of success and an unsuccessful node that outputs the probability of unsuccessful.

[0344] Fig.35 This is an explanatory diagram showing a screen displayed by the information processing device 20 of Embodiment 8. When receiving an instruction from an obstetrician to use sperm of a specific sex for microinsemination, the culture teacher as a user will use Fig.35 The screen for explanation is displayed on the display device 15. Displayed on the screen are an image column 61, a determination button 68, and an end button 67. In the image column 61, a microscope image captured by the camera 48 can be displayed in real time.

[0345] When the user selects the determination button 68, the control unit 21 extracts still image feature quantities and moving image feature quantities from the image displayed in the image column 61. The control unit 21 inputs the feature quantities into the gender determination learning model 56, and obtains the probability of being an X sperm and the probability of being a Y sperm. The control unit 21 displays the probability of being an X sperm and the probability of being a Y sperm in the gender column 82 at the upper right of the screen.

[0346] When the user selects the determination button 68 again, the control unit 21 updates the probability of being an X sperm and the probability of being a Y sperm using the image displayed in real time in the image field 61. When the user selects the end button 67, the control unit 21 ends the processing.

[0347] For example, by using YOLOv3 (You Only Look Once version 3) which is a type of CNN for the gender determination learning model 56, the control unit 21 can quickly obtain the probability of an X sperm and the probability of a Y sperm. When the probability of an X sperm and the probability of a Y sperm can be obtained sufficiently quickly, the control unit 21 can update the gender column 82 in real time. When the gender column 82 is updated in real time, the determination button 68 is not required.

[0348] Users use Fig.12 After finding sperm with a high probability of success in microinsemination, use Fig.35 The user can also determine the sex of sperm by using Fig.35 After finding the sperm of the sex specified by the obstetrician, use Fig.12 The success rate of microinsemination can be determined by the following instructions.

[0349] The information processing system 10 of the present embodiment may be a system independent of the information processing system 10 for determining the success probability of microinsemination described in the first embodiment and the like.

[0350] The control unit 21 can also use Fig.12 Description of the evaluation column 65 and use Fig.35 The gender columns 82 of the description are displayed side by side on one screen.

[0351] Fig.36 2 is a flowchart showing the processing flow of the program of the eighth embodiment. Fig.35 As shown, in a state where only one candidate sperm is displayed in the image field 61, the determination button 68 is selected.

[0352] The control unit 21 determines whether the determination button 68 is selected (step S701). If it is determined that the determination button 68 is selected (yes in step S701), the control unit 21 obtains an image of sperm through the camera 48 (step S702).

[0353] The control unit 21 extracts the sperm image feature value (step S703). The control unit 21 inputs the sperm image feature value into the gender determination learning model 56, and obtains the probability that the sperm under observation is an X sperm and the probability that it is a Y sperm (step S704). The control unit 21 displays the obtained probability in the gender column 82 (step S705).

[0354] When it is determined that the selection of the determination button 68 is not accepted (No in step S701), or after step S705 ends, the control unit 21 determines whether the selection of the end button 67 is accepted (step S706). When it is determined that the selection of the end button 67 is not accepted (No in step S706), the control unit 21 returns to step S701. When it is determined that the selection of the end button 67 is accepted (Yes in step S706), the control unit 21 ends the processing.

[0355] According to this embodiment, an information processing system 10 for determining and displaying the sex of sperm can be provided. The trainer can select sperm with a high probability of becoming the sex indicated by the obstetrician for microinsemination. Thus, the possibility of a child born through microinsemination suffering from a sex-linked genetic disease and the possibility of carrying a sex-linked genetic disease gene can be reduced. In addition, abortion caused by gene abnormalities associated with sex-linked genetic diseases can be prevented, and the probability of successful delivery after microinsemination can be increased.

[0356] [Implementation Method 9]

[0357] This embodiment relates to an information processing system 10 that displays the probability of a child having a sex-linked genetic disease and the probability of carrying a gene based on the result of determining whether the sperm under observation is an X sperm or a Y sperm. The description of the parts common to the eighth embodiment is omitted.

[0358] Fig.37 This is an explanatory diagram showing a screen displayed by the information processing device 20 according to the ninth embodiment. Fig.37 The screen shown shows a disease setting column 830, an egg donor column 832, and a sperm donor column 833. The disease setting column 830 includes a disease name column 831, an XY column 834, and a superiority column 835. The egg donor column 832 includes an egg identification information column 836 and an egg estimation information column 837. The sperm donor column 833 includes a sperm identification information column 838 and a sperm estimation information column 839. Fig.37 The screen shown is a setting input screen for inputting settings related to sex-linked genetic diseases.

[0359] A button for selecting the name of a sex-linked genetic disease to be determined is displayed in the disease name column 831. In the XY column 834 and the superiority column 835, the inheritance form corresponding to the sex-linked genetic disease selected in the disease name column 831 is displayed with a black circle.

[0360] Fig.37 In the example, “hemophilia” set in the disease name column 831 is a disease inherited through the X chromosome, and this fact is displayed in the XY column 834. Similarly, “hemophilia” is a sex-linked inferior genetic disease, which is displayed in the superior inferior column 835.

[0361] The user sets the name of the sex-linked genetic disease to be determined based on medical records, etc., in the disease name column 831. The control unit 21 displays the XY column 834 and the superiority or inferiority column 835 based on the database in which the disease name and the inheritance form are recorded in association.

[0362] In the case where the name of the sex-linked genetic disease is not clearly indicated in the medical record, etc. but only the inheritance form is indicated, the user does not use the disease name column 831, but inputs the inheritance form of the sex-linked genetic disease to be determined in the XY column 834 and the superiority column 835. In the case where the user wants to input the inheritance form, the white circle and the black circle displayed at the left end of the XY column 834 and the superiority column 835 respectively realize the input button function for accepting the user input.

[0363] The egg donor column 832 receives input of information related to the sex chromosomes of the egg donor. The user confirms whether the egg donor's sex chromosomes have abnormalities based on medical records or the like.

[0364] If confirmed, the user selects the check box arranged at the upper left of the egg confirmation information column 836 to display a check mark. The user uses the buttons displayed at the bottom of the egg confirmation information column 836 to input whether the two X chromosomes of the egg donor are normal or abnormal based on the medical records, etc. Fig.37 In the experiment, the subject that both X chromosomes are "abnormal" was entered.

[0365] If the determination is not confirmed, the user selects a check box arranged in the upper left of the egg estimation information column 837 to display a check mark. The egg estimation information column 837 displays columns for inputting the estimated probability of the egg donor's sex chromosome being normal, the estimated probability of being heterozygous abnormal, and the estimated probability of being homozygous abnormal in percentage.

[0366] If the sex chromosomes of the egg donor are not determined to be abnormal, experts such as obstetricians estimate the probability of the sex chromosomes of the egg donor being abnormal based on family history, etc., and record it in medical records, etc. The user inputs the estimated probability in each column of the egg estimation information column 837 based on medical records, etc.

[0367] The sperm donor column 833 receives input of information related to the sex chromosome of the sperm donor. The user confirms whether the sex chromosome of the sperm donor has been confirmed to be abnormal based on the medical records or the like.

[0368] If confirmed, the user selects the check box arranged at the upper left of the sperm confirmation information column 838 to display a check mark. The user uses the buttons displayed at the bottom of the sperm confirmation information column 838 to input whether the X chromosome and Y chromosome of the sperm donor are normal or abnormal, respectively, based on the medical records. Fig.37 In the data, the subject that both X chromosome and Y chromosome are "normal" was input.

[0369] If the determination is undecided, the user selects a check box arranged at the upper left of the sperm estimation information column 839 to display a check mark. The sperm estimation information column 839 displays columns for inputting the estimated probability of the X chromosome and Y chromosome of the sperm donor being normal and abnormal, respectively, in percentage.

[0370] If the sex chromosome of the sperm donor is not determined to be abnormal, experts such as obstetricians estimate the probability of the sex chromosome of the sperm donor being abnormal based on family history, etc., and record it in medical records, etc. The user inputs the estimated probability in each column of the sperm estimation information column 839 based on medical records, etc.

[0371] Specifically, when the XY column 834 displays that a determination is to be made regarding a disease inherited through the X chromosome, the user inputs the estimated probability that the X chromosome is normal and the estimated probability that it is abnormal based on the medical records, etc., into the sperm estimation information column 839. When the XY column 834 displays that a determination is to be made regarding a disease inherited through the Y chromosome, the user inputs the estimated probability that the Y chromosome is normal and the estimated probability that it is abnormal based on the medical records, etc., into the sperm estimation information column 839.

[0372] When numbers are input into two of the three columns included in the egg estimation information column 837, the control unit 21 may calculate the remaining one by setting the total probability to 1 and display it. Similarly, when numbers are input into one of the two columns included in the sperm estimation information column 839, the control unit 21 may calculate the other one by setting the total probability to 1 and display it. When the total probability of the egg estimation information column 837 and the total probability of the sperm estimation information column 839 are not 1, the control unit 21 may display an error message to urge the user to make corrections.

[0373] Fig.38 This is an explanatory diagram showing a screen displayed by the information processing device 20 according to the ninth embodiment. Fig.38 The screen shown replaces the Fig.35 The screen display of the instructions. Fig.38 In the illustrated screen, an incidence rate column 841 and a carrier rate column 842 are displayed between the sex column 82 and the determination button 68 .

[0374] The incidence rate column 841 shows the probability that a child born through microinsemination will suffer from a sex-linked genetic disease. The carrier rate column 842 shows the probability that a child born through microinsemination will be a carrier of a sex-linked genetic disease.

[0375] Taking the case of "X-deficient genetic disease" as an example, the method of calculating the incidence rate and carrier rate is briefly explained. The presence or absence of genetic abnormalities in the egg donor and sperm donor has a great impact on the incidence rate and carrier rate of the child. Fig.30 The relationship shown.

[0376] Fig.37 In the example shown, the egg donor's X chromosome is homozygous for the abnormal gene, and the sperm donor's X chromosome is normal. Fig.30 As shown in the rounded box on the upper right, if the child is a girl, the incidence rate is 0% and the carrier rate is 100%. If the child is a boy, the incidence rate and the carrier rate are both 100%.

[0377] Figure 30 to Figure 32 The table shown is recorded in the auxiliary storage device 23 as a database or a program that outputs the incidence rate and the carrier rate when a predetermined condition is input. The control unit 21 obtains the incidence rate and the carrier rate when the child is a boy and a girl, respectively, based on the user's input.

[0378] When sperm with a 10% probability of being X sperm and a 90% probability of being Y sperm are used for microinsemination, the probability that the fertilized egg is male, that is, the probability of having a boy, is 90%, and the probability that the fertilized egg is female, that is, the probability of having a girl, is 10%.

[0379] The control unit 21 calculates the morbidity rate of children born through microinsemination by adding the product of the probability of having a boy and the morbidity rate of boys to the product of the probability of having a girl and the morbidity rate of girls. Similarly, the control unit 21 calculates the carrier rate of children born through microinsemination by adding the product of the probability of having a boy and the carrier rate of boys to the product of the probability of having a girl and the carrier rate of girls.

[0380] Fig.37 In the example shown, the morbidity of children born by microinsemination is 90% and the carrier rate is 100%.

[0381] This is an example of a case where the egg donor's X chromosome is heterozygous for an abnormal gene, and the sperm donor's gene is normal. Fig.30 As shown in the rounded box in the upper center, if the child is a girl, the incidence rate is 0% and the carrier rate is 50%. If the child is a boy, the incidence rate and the carrier rate are both 50%.

[0382] When sperm with a 10% probability of being X sperm and a 90% probability of being Y sperm are used for microinsemination, the probability of having a boy is 90% and the probability of having a girl is 10%.

[0383] The control unit 21 calculates the disease incidence rate as 45% by adding the product of the boy's incidence rate of 50% and the probability of having a boy of 90% to the product of the girl's incidence rate of 0% and the probability of having a girl of 10%. Similarly, the control unit 21 calculates the disease carrier rate as 50%.

[0384] In use Fig.37 On the illustrated screen, when a value other than 0% or 100% is input in the egg estimation information column 837 or the sperm estimation information column 839 for the estimation probability, the control unit 21 calculates the disease incidence rate and carrier rate by weighted averaging the combinations of each case.

[0385] Fig.39 2 is a flowchart showing the processing flow of the program of the ninth embodiment. The control unit 21 displays the use Fig.37 The control unit 21 displays the setting input screen for the description (step S711). The control unit 21 obtains the setting input by the user (step S712).

[0386] The control unit 21 starts the morbidity and carrier rate calculation subroutine (step S713). The morbidity and carrier rate calculation subroutine is a subroutine that calculates the morbidity and carrier rate of sex-linked genetic diseases when X sperm is used and when Y sperm is used. The processing flow of the morbidity and carrier rate calculation subroutine will be described later.

[0387] The control unit 21 determines whether the determination button 68 is selected (step S701). If it is determined that the determination button 68 is selected (yes in step S701), the control unit 21 obtains an image of sperm through the camera 48 (step S702).

[0388] The control unit 21 extracts the sperm image feature value (step S703). The control unit 21 inputs the sperm image feature value into the sex determination learning model 56, and obtains the probability that the sperm under observation is an X sperm and the probability that it is a Y sperm (step S704).

[0389] The control unit 21 calculates the incidence rate when the sperm under observation is used based on the incidence rate of each sperm sex and the probability of each sperm sex acquired in step S713 (step S721). Specifically, the control unit 21 calculates the incidence rate using equation (7).

[0390] Incidence rate = Incidence rate when X sperm is used × Probability of X sperm

[0391] + The incidence rate when Y sperm is used × The probability of it being Y sperm... (7)

[0392] The control unit 21 calculates the carrier rate when using the sperm under observation based on the incidence rate of each sperm sex and the probability of each sperm sex acquired in step S713 (step S722). Specifically, the control unit 21 calculates the carrier rate using equation (8).

[0393] Carrier rate = Carrier rate when using X sperm × Probability of being X sperm

[0394] + Carrying rate when Y sperm is used × Probability of being Y sperm... (8)

[0395] The control unit 21 displays the incidence rate calculated in step S721 in the incidence rate column 841 , and displays the carrier rate calculated in step S722 in the carrier rate column 842 (step S723 ).

[0396] When it is determined that the selection of the determination button 68 is not accepted (No in step S701), or after step S723 is completed, the control unit 21 determines whether the selection of the end button 67 is accepted (step S706). When it is determined that the selection of the end button 67 is not accepted (No in step S706), the control unit 21 returns to step S701. When it is determined that the selection of the end button 67 is accepted (Yes in step S706), the control unit 21 ends the processing.

[0397] Fig.40 This is a flowchart showing the processing flow of the incidence rate and carrier rate calculation subroutine. The incidence rate and carrier rate calculation subroutine is a subroutine that calculates the incidence rate and carrier rate of sex-linked genetic diseases when X sperm is used and when Y sperm is used.

[0398] In the following description, Fig.37 The egg donor column 832 accepts input, and the probability of the egg donor's genes being normal is recorded as Pxn, the probability of heterozygous abnormalities is recorded as Pxt, and the probability of homozygous abnormalities is recorded as Pxm. Fig.37 The sperm donor column 833 receives input, and the probability of the sperm donor's genes being normal is recorded as Pyn, and the probability of abnormality is recorded as Pya.

[0399] When the egg identification information column 836 is used, any one of Pxn, Pxt and Pxm is 100%, and the others are 0%. When the sperm identification information column 838 is used, any one of Pyn and Pya is 100%, and the other is 0%.

[0400] The control unit 21 determines whether the sex-linked genetic disease inputted through the disease setting column 830 is a Y genetic disease (step S731). If it is determined to be a Y genetic disease (yes in step S731), the control unit 21 calculates the incidence rate and carrier rate of the sex-linked genetic disease in the fertilized egg, i.e., the newborn, when X sperm is used and when Y sperm is used (step S732).

[0401] Specifically, the incidence rate and carrier rate when X sperm is used are 0%. The incidence rate and carrier rate when Y sperm is used are equal to the probability Pya that the sex chromosome of the sperm donor is abnormal. Then, the control unit 21 ends the processing.

[0402] If it is determined that it is not a Y-linked genetic disease (No in step S731), the control unit 21 obtains the morbidity matrix Mp (step S733) related to the inheritance form of the sex-linked genetic disease input through the disease setting column 830. The morbidity matrix Mp is obtained by using Fig.30 and Fig.31 The incidence rates in the table showing the incidence rates and carrier rates of sex-linked genetic diseases described above are prepared according to the inheritance form and the sex of sperm, and recorded in the auxiliary storage device 23. The incidence rate matrix Mp is represented by equation (9).

[0403]

Number 7

[0404]

[0405] Mp11 represents the incidence rate when both the sex chromosomes of the sperm donor and the egg donor are normal.

[0406] Mp12 represents the incidence rate when the sex chromosome of the sperm donor is normal and the sex chromosome of the egg donor is heterozygous.

[0407] Mp13 represents the incidence rate when the sex chromosomes of the sperm donor are normal and the sex chromosomes of the egg donor are homozygous for abnormalities.

[0408] Mp21 indicates the incidence rate when the sex chromosome of the sperm donor is abnormal and the sex chromosome of the egg donor is normal.

[0409] Mp22 represents the incidence rate when the sperm donor has an abnormal sex chromosome and the egg donor is heterozygous for the abnormal sex chromosome.

[0410] Mp23 represents the incidence rate when the sperm donor has an abnormal sex chromosome and the egg donor is homozygous for the abnormal sex chromosome.

[0411] For example, regarding the use Fig.30Regarding the sex-linked hypogonadism described above, the incidence matrix Mp when X sperm is used is shown in equation (10), and the incidence matrix Mp when Y sperm is used is shown in equation (11).

[0412]

Number 8

[0413]

[0414] The control unit 21 calculates the incidence rate Pp associated with each of the X sperm and the Y sperm according to the formula (12) (step S734).

[0415]

Number 9

[0416]

[0417] The control unit 21 obtains a carrier rate matrix Mc (step S735) related to the inheritance form of the sex-linked genetic disease input through the disease setting column 830. The carrier rate matrix Mc is obtained by using Fig.30 and Fig.31 The carrier rate in the table showing the incidence and carrier rate of sex-linked genetic diseases described above is prepared according to the inheritance form and the sex of sperm, and is recorded in the auxiliary storage device 23. The carrier rate matrix Mc is represented by equation (13).

[0418]

Number 10

[0419]

[0420] Mc11 represents the carrier rate when the sex chromosomes of both the sperm donor and the egg donor are normal.

[0421] Mc12 represents the carrier rate when the sex chromosome of the sperm donor is normal and the sex chromosome of the egg donor is heterozygous.

[0422] Mc13 represents the carrier rate when the sex chromosome of the sperm donor is normal and the sex chromosome of the egg donor is homozygous.

[0423] Mc21 represents the carrier rate when the sex chromosome of the sperm donor is abnormal and the sex chromosome of the egg donor is normal.

[0424] Mc22 represents the carrier rate when the sperm donor has an abnormal sex chromosome and the egg donor has a heterozygous sex chromosome abnormality.

[0425] Mc23 represents the carrier rate when the sperm donor has an abnormal sex chromosome and the egg donor has a homozygous sex chromosome abnormality.

[0426] The control unit 21 calculates the carrying rate Pc for each of the X sperm and the Y sperm according to the formula (14) (step S736). Then, the control unit 21 ends the processing.

[0427]

Number 11

[0428]

[0429] According to the present embodiment, it is possible to provide an information processing system 10 that displays the incidence rate and carrier rate of sex-linked genetic diseases in addition to the sex of sperm.

[0430] According to the information provided by the obstetrician, the insemination specialist can select sperm with low incidence and carrier rate of sex-linked genetic diseases for microinsemination. As a result, the possibility of children born through microinsemination suffering from sex-linked genetic diseases and the possibility of carrying genes for sex-linked genetic diseases can be reduced. In addition, it can also prevent miscarriage caused by genetic abnormalities associated with sex-linked genetic diseases and increase the probability of successful delivery after microinsemination.

[0431] The incidence and carrier rates of multiple sex-linked genetic diseases can also be calculated and displayed in a list. In the case where the egg donor and sperm donor each have genetic abnormalities related to different sex-linked genetic diseases, sperm predicted to have a low incidence and carrier rate of sex-linked genetic diseases can be selected for microinsemination.

[0432] The technical features (structural elements) described in the embodiments can be combined with each other, and new technical features can be formed by the combination.

[0433] The embodiments disclosed this time should be considered as illustrative in every respect and not restrictive. The scope of the present invention is indicated by the claims rather than the above meaning, and includes all modifications within the meaning equivalent to the claims and the scope thereof.

[0434] Description of Reference Numerals

[0435] 10 Information Processing System

[0436] 15 Display device

[0437] 20Information processing device

[0438] 21 Control Department

[0439] 22 Main storage device

[0440] 23 Auxiliary storage device

[0441] 24 Ministry of Communications

[0442] 25 Display I / F

[0443] 26 Stage I / F

[0444] 28 Shooting I / F

[0445] 29 Reading unit

[0446] 41 Microscope

[0447] 42 stage

[0448] 421 Observation Container

[0449] 43 Eyepiece

[0450] 44 Lighting Department

[0451] 45 optical path division unit

[0452] 46 stage moving part

[0453] 47 objective lens

[0454] 48 Cameras

[0455] 51 training data DB

[0456] 52 Preparatory shooting DB

[0457] 53 Process Learning Model (Learning Model)

[0458] 531 Input layer

[0459] 532 Middle Layer

[0460] 533 output layer

[0461] 54 Image feature model

[0462] 541 Input layer

[0463] 542 Middle Layer

[0464] 543 Output layer

[0465] 545 Central Floor

[0466] 546 Image Encoder

[0467] 56 Gender determination learning model

[0468] 561 Input layer

[0469] 562 Middle Layer

[0470] 563 Output layer

[0471] 57 Normal sperm determination model

[0472] 571 Input layer

[0473] 572 Middle Layer

[0474] 573 Output layer

[0475] 58 gender training data DB

[0476] 61 Image Bar

[0477] 62 Target Number Column

[0478] 63 Number of columns shot

[0479] 64 Patient Information Column

[0480] 65 Evaluation column

[0481] 651 First Evaluation Column

[0482] 652 Second Evaluation Column

[0483] 653 Third Evaluation Column

[0484] 654 Fourth Evaluation Column

[0485] 655 Fifth Evaluation Column

[0486] 659 Comprehensive Evaluation Column

[0487] 66 Shooting button

[0488] 67 End button

[0489] 68 judgment button

[0490] 69Next button

[0491] 71 Photographed image acquisition unit

[0492] 72 Input unit

[0493] 73 Output unit

[0494] 81 indicator line

[0495] 82 Gender Column

[0496] 830 Disease Setting Column

[0497] 831 Disease Name Column

[0498] 832 Egg Provider Column

[0499] 833 Sperm provider column

[0500] 834XY column

[0501] 835 Advantages and Disadvantages Column

[0502] 836 Egg confirmation information column

[0503] 837 Egg Estimation Information Column

[0504] 838 Sperm confirmation information column

[0505] 839 sperm estimation information column

[0506] 841 Incidence column

[0507] 842 Carry Rate Column

[0508] 90 Computer

[0509] 96 Removable recording medium

[0510] 97 Programs

[0511] 98 Semiconductor Memory

Claims

1. A removable recording medium, It is characterized in that A program is recorded that causes a computer to execute the following processing: acquiring a plurality of images in which candidate sperm are captured, the candidate sperm being contained in semen extracted from a sperm donor who wishes to undergo microinsemination, The acquired captured images of the candidate sperm are input into a learning model, the learning model receives the captured images of the candidate sperm and outputs a prediction related to whether the candidate sperm grows normally to each stage after microinsemination using the candidate sperm, the learning model is a learned model that has been machine-learned using training data, the training data recording the captured images of sperm used in microinsemination in the past and whether the sperm grows normally to each stage after microinsemination using the sperm in association with each other, Based on the input image of the candidate sperm, a prediction of whether the sperm has grown normally to each stage is obtained, which is output from the learning model. A sampling distribution involving success or failure of each stage is generated based on the predictions obtained.

2. The portable recording medium according to claim 1, in, The learning model includes an image encoder for extracting image features from an input image in which a candidate sperm is captured. The prediction is outputted based on the image feature amount extracted by the image encoder.

3. The portable recording medium according to claim 1, in, The prediction is the predicted success probability at each stage from normal growth to microinsemination.

4. The portable recording medium according to claim 1, in, A first distribution of whether or not the cells have grown to each stage after microinsemination in a plurality of microinsemination cases performed in the past is obtained, A second distribution is generated based on the first distribution and the sampling distribution.

5. The portable recording medium according to claim 4, in, acquiring an image of a candidate sperm contained in semen extracted from the sperm donor, obtaining a prediction output by inputting a captured image containing a candidate sperm into the learning model, An evaluation corresponding to the prediction in the second distribution is output.

6. The portable recording medium according to claim 5, in, The reliability of the evaluation is output.

7. The portable recording medium according to claim 1, in, Obtain the success or failure of each stage after microinsemination. Additional data is recorded that associates an image of sperm used in the microinsemination with the success or failure of each stage obtained.

8. The portable recording medium according to claim 7, in, The learning model is updated using the additional data.

9. The portable recording medium according to claim 8, in, The updated learning model is sent to a microinsemination implementation institution.

10. The portable recording medium according to any one of claims 1 to 9, in, Get a microscope image taken through a microscope, determining whether the obtained microscope image contains a candidate sperm, When it is determined that the candidate sperm is contained, the microscope image is used as the captured image in which the candidate sperm is captured.

11. The portable recording medium according to claim 10, in, A portion of the microscope image containing the candidate sperm is selected as a captured image in which the candidate sperm is captured.

12. The portable recording medium according to any one of claims 1 to 9, in, The acquired image of the candidate sperm is input into a sex determination learning model, which receives the image of the candidate sperm and outputs a prediction related to the sex chromosome contained in the candidate sperm. The output is a prediction output by the gender determination learning model based on the input captured image.

13. The portable recording medium according to claim 12, in, Get the first information about sex-linked genetic diseases of egg and sperm donors, obtaining second information related to the genetic form of the sex-linked genetic disease, Based on the incidence of the sex-linked genetic disease obtained from the first information and the second information, and the prediction obtained by the gender determination learning model, the probability that a child born through microinsemination will suffer from the sex-linked genetic disease is output.

14. The removable recording medium according to claim 12, in, Get the first information about sex-linked genetic diseases of egg and sperm donors, obtaining second information related to the genetic form of the sex-linked genetic disease, Based on the carrier rate of the sex-linked genetic disease obtained from the first information and the second information, and the prediction obtained by the gender determination learning model, the probability that a child born through microinsemination has a gene for the sex-linked genetic disease is output.

15. An information processing device, It is characterized in that include: a captured image acquisition unit that acquires a plurality of captured images of candidate sperm contained in semen extracted from a sperm donor who wishes microinsemination; inputting the acquired captured images of the candidate sperm into an input unit of a learning model, the learning model receiving the captured images of the candidate sperm and outputting a prediction related to whether the candidate sperm grows normally to each stage after microinsemination, the learning model being a learned model that has been machine-learned using training data that records captured images of sperm used in microinsemination in the past in association with whether the sperm grows normally to each stage after microinsemination using the sperm; A prediction acquisition unit that acquires a prediction outputted from the learning model regarding whether the sperm has grown normally to each stage, based on an input captured image in which the candidate sperm is captured; and An output section is provided for outputting a sampling distribution of success or failure of each stage based on the obtained predictions.

16. A method of information processing, It is characterized in that The computer is made to execute the following processing: acquiring a plurality of images in which candidate sperm are captured, the candidate sperm being contained in semen extracted from a sperm donor who wishes to undergo microinsemination, The acquired captured images of the candidate sperm are input into a learning model, the learning model receives the captured images of the candidate sperm and outputs a prediction related to whether the candidate sperm grows normally to each stage after microinsemination using the candidate sperm, the learning model is a learned model that has been machine-learned using training data, the training data recording the captured images of sperm used in microinsemination in the past and whether the sperm grows normally to each stage after microinsemination using the sperm in association with each other, Based on the input image of the candidate sperm, a prediction of whether the sperm has grown normally to each stage is obtained, which is output from the learning model. A sampling distribution involving success or failure of each stage is generated based on the predictions obtained.

17. A method for displaying information, It is characterized in that photographing a plurality of candidate spermatozoa contained in semen extracted from a sperm donor desiring microinsemination, The image captured by the candidate sperm is input into a learning model, the learning model receives the image captured by the candidate sperm, and outputs a prediction related to whether the candidate sperm grows normally to each stage after microinsemination using the candidate sperm, the learning model is a learned model that has been machine-learned using training data, and the training data records the image captured by the sperm used in microinsemination in the past and whether the sperm grows normally to each stage after microinsemination using the sperm, in association with each other, The sampling distribution related to success or failure of each stage generated based on the prediction of whether the sperm grows normally to each stage is displayed. The prediction is output by the learning model based on the input image of the candidate sperm.

18. A method for making a learning model, It is characterized in that For generating a sampling distribution related to the success or failure of microinsemination based on a plurality of captured images of candidate sperm contained in semen extracted from a sperm donor who wishes to undergo microinsemination, the manufacturing method comprising: Acquire training data that records images of sperm used in microinsemination and correlates the images of sperm used in microinsemination with the stages of whether the sperm can grow normally after microinsemination with the sperm, for a plurality of microinsemination cases performed in the past. A learning model is generated, the learning model taking an image of a candidate sperm as input and whether the sperm can grow normally to each stage after micro-insemination as output, and when the learning model is input with an image of a candidate sperm to be used in micro-insemination, the learning model outputs a prediction on whether the sperm can grow normally to each stage when micro-insemination is performed using the candidate sperm.