Method for Determining Developmental Stage of Fertilized Egg, Program, Recording Medium, Imaging Method, and Imaging Apparatus

Abstract

To objectively and accurately determine the developmental stage of a fertilized egg using image data obtained by imaging the fertilized egg with an optical microscope.SOLUTION: A determination method for the developmental stage of a fertilized egg according to the present invention comprises: obtaining image data corresponding to an image of a fertilized egg captured by an optical microscope; extracting a region corresponding to the fertilized egg in the image based on the image data and obtaining an index value representing the degree of matching to the circle of the region; extracting a region corresponding to the zona pellucida of the fertilized egg in the image based on the image data and obtaining an index value representing the thickness of the region; and determining the developmental stage of the fertilized egg based on the index value representing the degree of matching, the index value representing the thickness, and a predetermined criterion for the combination thereof.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] This invention relates to a technique for determining the developmental stage of a fertilized egg using image data obtained by imaging the fertilized egg.

Background Art

[0002] For example, in assisted reproductive medicine for the purpose of infertility treatment, embryos (fertilized eggs) that have been fertilized outside the body and cultured for a certain period are returned to the body. However, the pregnancy success rate in such (assisted reproductive medicine) is not necessarily high, and the mental and economic burdens on patients are also significant. To solve this problem, methods for accurately determining the state of the cultured embryos are being explored.

[0003] Conventionally, for the evaluation of whether embryo culture is progressing well, it is common for doctors and embryologists to visually perform it by, for example, microscopic observation. As judgment criteria, for example, Veeck classification and Gardner classification are widely used, but these only show approximate judgment criteria for the morphological characteristics of embryos, and the current situation is that the final evaluation depends on the subjective judgment of the evaluator. For this reason, a technique that enables objective and quantitative evaluation is required.

[0004] To meet such requirements, the applicant of the present application previously disclosed Patent Documents 1 and 2. These patent documents describe a method for identifying and separating the trophectoderm and the inner cell mass from a three-dimensional image of an embryo (fertilized egg) imaged by a non-invasive tomographic imaging technique such as optical coherence tomography (OCT).

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] In observing the state of a fertilized egg, especially its changes, it is necessary to repeatedly perform imaging at regular time intervals. However, for example, compared with optical microscope imaging, OCT imaging requires a long imaging time. Therefore, especially when observing a plurality of fertilized eggs in parallel, if all of them are to be subjected to OCT imaging, there is a problem that the imaging interval becomes long. From this, it is difficult to always perform OCT imaging on all fertilized eggs in order, and it is realistic to preferentially image the fertilized eggs that require follow-up observation. Here, the fertilized eggs to be preferentially imaged are, for example, fertilized eggs in a transitional period when the developmental stage changes.

[0007] As a means for determining the developmental stage of a fertilized egg, a method of quantifying and evaluating the trophoblast, inner cell mass, zona pellucida, etc., which are components of the fertilized egg, by image analysis can be considered. However, at present, a method for accurately determining the stage from the luminance information of OCT images has not been established. In addition, as described above, since OCT imaging takes time, it cannot be said to be suitable for applications in which it is required to determine the stage of a large number of fertilized eggs respectively.

[0008] In order to solve these problems, it is desired to establish a method for objectively and accurately determining the developmental stage of a fertilized egg from images periodically captured by, for example, optical microscope imaging, which enables imaging in a shorter time.

[0009] This invention has been made in view of the above problems, and an object thereof is to provide a technique capable of objectively and accurately determining the developmental stage of a fertilized egg using image data obtained by imaging the fertilized egg with an optical microscope.

Means for Solving the Problems

[0010] One aspect of the present invention is to image a fertilized egg under an optical microscope in order to achieve the above object. One A step of obtaining image data corresponding to the image, a step of extracting a region corresponding to the fertilized egg in the image based on the image data, and obtaining an index value representing the degree of coincidence of the region with a circle, and based on the image data, A step of extracting a region corresponding to the zona pellucida of the fertilized egg in the image and obtaining an index value representing the thickness of the region, and based on the index value representing the degree of coincidence and the index value representing the thickness, and a predetermined determination criterion for these combinations, A method for determining the developmental stage of a fertilized egg, comprising a step of determining the developmental stage of the fertilized egg.

[0011] In the invention configured as described above, in the developmental stage of the fertilized egg after the blastocyst stage, first, the zona pellucida becomes thinner (expanded blastocyst stage), and then the internal cells protrude from the zona pellucida (hatching). Focusing on the phenomenon, the developmental stage of the fertilized egg is determined from the optical microscope image. Specifically, the developmental stage is determined by a combination of the index value of the thickness of the region corresponding to the zona pellucida extracted from the image and the index value representing how close the outer shape of the fertilized egg is to a circle. Although it will be described in detail later, by automatically performing the determination using the index value quantified in view of the morphological characteristics of the fertilized egg at each developmental stage in this way, an objective and accurate determination becomes possible.

[0012] Another aspect of the present invention is a program for causing a computer to execute each step of the above method, and a computer-readable recording medium on which the program is recorded. The above method is suitable for execution by a computer device, and by realizing it as a program, it becomes possible to perform the determination using existing hardware resources.

[0013] Another aspect of the present invention is an imaging method comprising a step of determining the developmental stage of the fertilized egg by the above-described method for determining the developmental stage of the fertilized egg, and a step of performing optical coherence tomography imaging of the fertilized egg according to the result of the determination.

[0014] Another aspect of the present invention is a two-dimensional image acquisition unit that captures a fertilized egg with an optical microscope to acquire two-dimensional image data, a three-dimensional image acquisition unit that captures the fertilized egg with optical coherence tomography to acquire three-dimensional image data, based on the two-dimensional image data, extracting a region corresponding to the fertilized egg in the image, obtaining an index value representing the degree of coincidence with a circle of the region, and Corresponding to one image of the fertilized egg captured based on the two-dimensional image data, extracting a region corresponding to the zona pellucida of the fertilized egg in the image, obtaining an index value representing the thickness of the region, and based on the index value representing the degree of coincidence and the index value representing the thickness, and a determination criterion predetermined for these combinations, an image processing unit that determines the developmental stage of the fertilized egg, and an imaging apparatus that performs optical coherence tomography imaging of the fertilized egg by the three-dimensional image acquisition unit according to the result of the determination by the image processing unit.

[0015] In the invention configured as described above, the developmental stage of the fertilized egg is determined from the optical microscope image, and optical coherence tomography imaging, that is, OCT imaging, is executed according to the result. In this way, by determining the stage of the fertilized egg from the image obtained by optical microscope imaging that can be performed in a short time, and from the result, executing OCT imaging capable of obtaining a detailed three-dimensional image of the fertilized egg at the necessary timing, it becomes possible to efficiently observe the state change of the fertilized egg.

Effect of the Invention

[0016] As described above, according to the present invention, by using the image obtained by optical microscope imaging that can be performed in a short time and performing quantification based on the characteristic shape at the developmental stage of the fertilized egg, it is possible to objectively and accurately determine the developmental stage of the fertilized egg. Also, by using the determination result, it becomes possible to accurately grasp the timing at which OCT imaging should be performed.

Brief Description of the Drawings

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Mode for Carrying Out the Invention

[0018] FIG. 1 is a diagram showing a configuration example of an image processing apparatus suitable as an execution entity of the present invention. This image processing apparatus 1 tomographically images a sample carried in a liquid, for example, an embryo (fertilized egg) cultured in a culture solution, and processes the obtained tomographic image to create a cross-sectional image showing the structure of one cross-section of the sample. Also, a three-dimensional image of the sample is created from a plurality of cross-sectional images. In order to uniformly show the directions in the following figures, as shown in FIG. 1, an XYZ orthogonal coordinate axis is set. Here, the XY plane represents the horizontal plane. Also, the Z axis represents the vertical axis, and more specifically, the (-Z) direction represents the vertically downward direction.

[0019] The image processing apparatus 1 includes a holding unit 10. The holding unit 10 holds a sample container 11 that houses a sample S to be imaged in a horizontal posture. The sample container 11 is, for example, a flat-bottomed and shallow dish-shaped container called a dish in which a recess capable of carrying a liquid is formed on the upper surface of a plate-like member. A medium M such as a culture solution is injected into the sample container 11, and a fertilized egg as the sample S is carried therein.

[0020] In this example, a plurality of samples S are carried on the sample container 11 having a single recess, but the present invention is not limited to this. For example, the sample container 11 may be a well plate in which a plurality of recesses called wells are arranged in a single plate-shaped member. In this case, the plurality of samples S can be carried one by one in the plurality of wells, respectively. Further, for example, a plurality of dishes each carrying a sample S may be held by the holding unit 10 in a horizontally arranged state and subjected to imaging.

[0021] Below the sample container 11 held by the holding unit 10, the imaging unit 20 is arranged. As the imaging unit 20, an optical coherence tomography (OCT) device capable of non-contact and non-destructive (non-invasive) imaging of a tomographic image of an object to be imaged is used. Although details will be described later, the imaging unit 20 which is an OCT device includes a light source 21 that generates illumination light for the object to be imaged, an optical fiber coupler 22, an object optical system 23, a reference optical system 24, a spectroscope 25, and a photodetector 26.

[0022] The imaging unit 20 further includes a microscope imaging unit 28 for performing optical microscope imaging. More specifically, the microscope imaging unit 28 includes an imaging optical system 281 and an imaging element 282. The imaging optical system 281 includes an objective lens whose focus is adjusted to the sample S in the sample container 11. As the imaging element 282, for example, a CCD imaging element, a CMOS sensor, or the like can be used. As the microscope imaging unit 28, one capable of bright-field imaging or phase-contrast imaging is preferable. The object optical system 23 and the microscope imaging unit 28 are supported by a support member (not shown) movable in the horizontal direction, and the position in the horizontal direction can be changed.

[0023] The image processing apparatus 1 further includes a control unit 30 that controls the operation of the apparatus, and a drive unit 40 that drives the movable part of the imaging unit 20. The control unit 30 includes a CPU (Central Processing Unit) 31, an A / D converter 32, a signal processing unit 33, an imaging control unit 34, an interface (IF) unit 35, an image memory 36, and a memory 37.

[0024] The CPU 31 controls the operation of the entire apparatus by executing a predetermined control program, and the control program executed by the CPU 31 and the data generated during processing are stored in the memory 37. The A / D converter 32 converts the signal output according to the received light amount from the photodetector 26 and the imaging element 282 of the imaging unit 20 into digital data. The signal processing unit 33 performs signal processing described later based on the digital data output from the A / D converter 32 to create an image of the object to be imaged. The image data thus created is appropriately stored and saved by the image memory 36.

[0025] The imaging control unit 34 controls each part of the imaging unit 20 to execute imaging processing. Specifically, the imaging control unit 34 selectively positions the object optical system 23 for OCT imaging and the microscope imaging unit 28 for optical microscope imaging at a predetermined imaging position where the imaging sample S is placed within the imaging field of view. When the object optical system 23 is positioned at the imaging position, the imaging control unit 34 causes the imaging unit 20 to execute OCT imaging processing described later to obtain three-dimensional image data representing the three-dimensional structure of the sample S. On the other hand, when the microscope imaging unit 28 is positioned at the imaging position, the imaging control unit 34 causes the microscope imaging unit 28 to obtain two-dimensional image data representing the planar image of the sample S formed on the light-receiving surface of the imaging element 282 by the imaging optical system 281.

[0026] The interface unit 35 is responsible for the communication between the image processing apparatus 1 and the outside. Specifically, the interface unit 35 has a communication function for communicating with external devices and a user interface function for receiving operation inputs from the user and notifying the user of various types of information. For this purpose, the interface unit 35 is connected to an input device 351 such as a keyboard, a mouse, or a touch panel that can receive operation inputs related to function selection and operation condition setting of the apparatus, and a display unit 352 composed of, for example, a liquid crystal display that displays various processing results such as tomographic images created by the signal processing unit 33 and three-dimensional images created by the 3D restoration unit 34.

[0027] In the imaging unit 20, a low-coherence light beam including broadband wavelength components is emitted from a light source 21 having a light-emitting element such as a light-emitting diode or a superluminescent diode (SLD). For the purpose of imaging a sample such as a cell, it is preferable to use, for example, near-infrared light in order to allow the incident light to reach the inside of the sample.

[0028] The light source 21 is connected to an optical fiber 221 that is one of the optical fibers constituting the optical fiber coupler 22, and the low-coherence light emitted from the light source 21 is branched by the optical fiber coupler 22 into light to two optical fibers 222 and 224. The optical fiber 222 constitutes an object system optical path. More specifically, the light emitted from the end of the optical fiber 222 enters the object optical system 23.

[0029] The object optical system 23 includes a collimator lens 231 and an objective lens 232. The light emitted from the end of the optical fiber 222 enters the objective lens 232 through the collimator lens 231. The objective lens 232 has a function of converging the light (observation light) from the light source 21 onto the sample and a function of condensing the reflected light emitted from the sample and directing it toward the optical fiber coupler 22. Although a single objective lens 232 is shown in the figure, a plurality of optical elements may be combined. The reflected light from the object to be imaged enters the optical fiber 222 as signal light through the objective lens 232 and the collimator lens 231. The optical axis of the objective lens 232 is orthogonal to the bottom surface of the sample container 11, and in this example, the optical axis direction coincides with the vertical axis direction.

[0030] The CPU 31 gives a control command to the imaging control unit 34, and in response, the imaging control unit 34 causes the imaging unit 20 to move in a predetermined direction. More specifically, the imaging control unit 34 moves the imaging unit 20 in the horizontal direction (XY direction) and the vertical direction (Z direction). By moving the imaging unit 20 in the horizontal direction, the imaging range changes in the horizontal direction. Also, by moving the imaging unit 20 in the vertical direction, the focal position in the optical axis direction of the objective lens 232 changes with respect to the sample S, which is the object to be imaged.

[0031] A part of the light incident from the light source 21 on the optical fiber coupler 22 enters the reference optical system 24 through the optical fiber 224. The reference optical system 24 includes a collimator lens 241 and a reference mirror 243, and these together with the optical fiber 224 constitute a reference optical path. Specifically, the light emitted from the end of the optical fiber 224 enters the reference mirror 243 through the collimator lens 241. The light reflected by the reference mirror 243 enters the optical fiber 224 as reference light.

[0032] The reference mirror 243 is supported by a reciprocating member (not shown) that operates according to a control command from the imaging control unit 34 and is movable back and forth in the Y direction. By moving the reference mirror 243 in the Y direction, that is, in the direction of approaching and separating from the collimator lens 241, the optical path length of the reference light reflected by the reference mirror 243 is adjusted.

[0033] The reflected light (signal light) reflected by the surface or internal reflecting surface of the sample and the reference light reflected by the reference mirror 243 are mixed by the optical fiber coupler 22 and enter the photodetector 26 through the optical fiber 226. At this time, interference caused by the phase difference occurs between the signal light and the reference light, and the spectral spectrum of the interference light varies depending on the depth of the reflecting surface. That is, the spectral spectrum of the interference light has information in the depth direction of the object to be imaged. Therefore, by spectroscopically analyzing the interference light for each wavelength, detecting the amount of light, and performing Fourier transform on the detected interference signal, the reflected light intensity distribution in the depth direction of the object to be imaged can be obtained. The OCT imaging technology based on such a principle is called Fourier Domain OCT (FD-OCT).

[0034] In the imaging unit 20 of this embodiment, a spectroscope 25 is provided on the optical path of the interference light from the optical fiber 226 to the photodetector 26. As the spectroscope 25, for example, one using a prism, one using a diffraction grating, etc. can be used. The interference light is spectroscopically analyzed for each wavelength component by the spectroscope 25 and received by the photodetector 26.

[0035] By performing Fourier transform on the interference signal output from the photodetector 26 in response to the interference light detected by the photodetector 26, the reflected light intensity distribution in the depth direction, that is, the Z direction, at the incident position of the illumination light in the sample is obtained. By scanning the light beam incident on the sample container 11 in the X direction, the reflected light intensity distribution in a plane parallel to the XZ plane is obtained, and a tomographic image of the sample with that plane as a cross section can be created. Since the principle is well known, detailed description is omitted.

[0036] Also, by changing the beam incident position in the Y direction in multiple steps and taking tomographic images each time, a large number of tomographic images can be obtained by tomographically imaging the sample in a cross-section parallel to the XZ plane. If the scanning pitch in the Y direction is reduced, image data with sufficient resolution to grasp the three-dimensional structure of the sample can be obtained. Three-dimensional image data (so-called voxel data) corresponding to the three-dimensional image of the sample can be created from these tomographic image data.

[0037] Thus, this image processing apparatus 1 has a function of acquiring an image of the sample S carried together with the culture medium M in the sample container 11. As the image, two-dimensional image data obtained by optical microscope imaging, tomographic image data obtained by OCT imaging, and three-dimensional image data based thereon can be acquired.

[0038] Hereinafter, one aspect of the image processing executable using the image processing apparatus 1 configured as described above will be described. This image processing corresponds to one embodiment of the fertilized egg development stage determination method and imaging method according to the present invention. The image processing in this embodiment is a process for acquiring a three-dimensional image of a fertilized egg around the blastocyst stage (hereinafter simply referred to as "embryo") as the sample S at a necessary timing.

[0039] For example, as known as Gardner classification, the development process of the embryo is classified into several development stages, and a three-dimensional image is particularly useful in observing and evaluating the embryo immediately after transitioning from one stage to the next. Since OCT imaging can obtain a three-dimensional image of the embryo to be evaluated, it is suitable for observing and evaluating the embryo from various directions in a multi-faceted manner, but it takes a relatively long time (for example, several minutes) for imaging. Therefore, when imaging and evaluating a plurality of embryos in order, the imaging interval for one embryo becomes long, and there may be a problem that a three-dimensional image cannot be acquired at a necessary timing.

[0040] Therefore, in this embodiment, the microscope imaging unit 28 capable of acquiring an image in a shorter time performs time-lapse imaging of the embryo at regular time intervals, and the CPU 31 determines the developmental stage of the embryo based on the obtained images. For the embryos determined to be necessary from the results, the imaging unit 20 performs OCT imaging.

[0041] A series of processes is executed by the control unit 30, more specifically, the CPU 31 executing a control program prepared in advance to cause each part of the apparatus to perform a predetermined operation. The control program can be implemented in the control unit 30 by reading it from an appropriate recording medium that non-temporarily records the control program, or by receiving it via a telecommunication line.

[0042] FIG. 2 is a diagram schematically showing the structure of an embryo as a sample in this embodiment. As is already known, when an egg is fertilized, cleavage begins, and a blastocyst is formed through a state called a morula. FIG. 2(a) schematically shows an embryo E1 in the morula stage. The embryo E1 in this state has a structure in which a mass of a large number of cells C generated by cleavage is surrounded by a zona pellucida ZP, which is a membrane having a substantially uniform thickness mainly composed of glycoprotein.

[0043] FIG. 2(b) schematically shows an embryo E2 in the early blastocyst stage. In the early blastocyst stage, a cavity called a blastocoel B is formed inside the embryo E2. More specifically, the cells C that have undergone cleavage are arranged in layers on the surface of the embryo (inside the zona pellucida ZP) to form a trophectoderm T, and the internal space surrounded by the trophectoderm T forms the blastocoel B.

[0044] The trophectoderm T has various thicknesses depending on the position and is distributed so as to adhere to the entire inner surface of the zona pellucida ZP. However, as it further grows, like the embryo E3 shown in FIG. 2(c), the trophectoderm T becomes a thin layer generally composed of one layer of cells C, and an inner cell mass I in which a large number of cells are concentrated in part is formed (blastocyst stage). As growth progresses, the blastocoel B expands and the embryo E3 grows, while the zona pellucida ZP becomes thinner (expanded blastocyst stage).

[0045] Then, as in the embryo E4 shown in Fig. 2(d), the escape (hatching) of the inner cells from the zona pellucida ZP begins (the escaped blastocyst stage), and finally all the cells escape and implant in the uterus, establishing pregnancy. In assisted reproductive medicine, assisted hatching that artificially induces hatching by external stimulation is also performed.

[0046] In the image processing of this embodiment, the developmental stages from approximately the early blastocyst stage to the escaped blastocyst stage, mainly centered on the expanded blastocyst stage, are automatically determined from the optical microscope images. First, the principle will be explained, and then the specific processing content will be described. In the following description, "before the expanded blastocyst stage" is a general concept that collectively refers to the developmental stages before the expanded blastocyst stage (e.g., the morula stage, the early blastocyst stage), and does not include the expanded blastocyst stage. Also, "after the expanded blastocyst stage" is a general concept that collectively refers to the developmental stages after the expanded blastocyst stage (e.g., the escaped blastocyst stage, the hatched blastocyst stage), and does not include the expanded blastocyst stage. Also, in the following, the "developmental stage" may be simply abbreviated as the "stage".

[0047] In the bright-field image or phase-contrast image obtained by optical microscope imaging, it is relatively easy to distinguish the zona pellucida from other structures based on the differences in its brightness and texture. And in well-cultured embryos, the zona pellucida generally has a uniform thickness. From the early stage of development to the early blastocyst stage, before the expanded blastocyst stage, there is no significant change in the thickness of the zona pellucida ZP, but in the expanded blastocyst stage, the zona pellucida ZP becomes thinner. From this, it is expected that the expanded blastocyst stage and the stages before it can be distinguished using the thickness of the zona pellucida appearing in the image as an index.

[0048] On the other hand, in the discrimination between the expanded blastocyst stage and subsequent stages, the thickness of the zona pellucida ZP is not an effective indicator. For this discrimination, it is conceivable to use the outer shape of the embryo as an indicator. From the initial stage of development to the expanded blastocyst stage, the outer shape of the embryo is almost spherical and almost circular in the image. In contrast, after hatching begins, the outer shape of the embryo deviates significantly from a sphere. Therefore, it is expected that the expanded blastocyst stage and subsequent stages can be discriminated by using as an indicator how close the outer shape of the embryo is to a circle in the image.

[0049] From the above, the inventor of the present application conceived the technical idea of automatically determining at which developmental stage the embryo is, before the expanded blastocyst stage, the expanded blastocyst stage, or after the expanded blastocyst stage, by combining the index value representing the thickness of the zona pellucida ZP and the index value representing the outer shape of the embryo, more specifically, the degree of conformity to a circle. As a preparatory step, a large number of images of embryos whose developmental stages were determined by skilled embryologists were collected, and the correlation between the above index values obtained from these images and the determination results was investigated. The results are shown below.

[0050] Figure 3 is a diagram showing the distribution of the zona pellucida thickness of the embryo. Regarding the thickness of the zona pellucida ZP, it was measured at multiple locations for one embryo, and the median or average value of these measurement values was used as the index value representing the thickness of the zona pellucida of the embryo. There are significant individual differences in the thickness of the zona pellucida ZP from about 3 μm to 20 μm or more depending on the embryo. However, as shown in Figure 3, there is a remarkable tendency that the zona pellucida ZP is relatively thick at the expanded blastocyst stage and is distributed in a thinner range before that. The boundary is approximately 13 μm. Therefore, by evaluating the thickness of the zona pellucida ZP using this value as the threshold T1, it is possible to discriminate between the expanded blastocyst stage and before that.

[0051] Figure 4 is a diagram showing the distribution of the aspect ratio of embryos. As an index value representing the degree of coincidence between the shape of the embryo in the image and a circle, various values can be considered. Here, the circumscribed rectangle of the embryo, that is, the rectangle with the minimum area among the rectangles circumscribing the periphery of the embryo, is specified, and it is represented by the aspect ratio, which is the ratio of the short side to the long side. If the embryo is circular, the circumscribed rectangle becomes a square and the aspect ratio is 1. Otherwise, the aspect ratio is less than 1, and the more the shape of the embryo deviates from a circle, the smaller the aspect ratio becomes. Therefore, the aspect ratio defined in this way can represent the degree of coincidence between the shape of the embryo and a circle.

[0052] The operation for specifying the circumscribed rectangle of the image object is implemented as a standard function in general image processing software. Therefore, using the aspect ratio obtained from the result as an index value of the shape of the embryo facilitates the processing. It is expected that similar results can be obtained by using, for example, the circularity obtained for the contour of the embryo as the index value.

[0053] As shown in Figure 4, in the expanded blastocyst stage and earlier stages, the aspect ratio is generally close to 1, and there is no significant difference between the two stages. On the other hand, in embryos after the expanded blastocyst stage, the center of the distribution spreads to a region with a lower aspect ratio.

[0054] Figure 5 is a diagram showing the distribution of embryos when two index values are combined. More specifically, Figure 5 is a scatter plot in which the positions occupied by each embryo are plotted according to these index values in a two-dimensional coordinate plane with two index values (zona pellucida thickness, aspect ratio) as the horizontal axis and the vertical axis, respectively. It can also be said that it is a diagram representing the distribution in the two-dimensional feature amount space when the two index values are regarded as feature amounts respectively.

[0055] From this figure, it can be seen that embryos before the expanded blastocyst stage indicated by circles are concentrated in the upper right of the figure, embryos at the expanded blastocyst stage indicated by triangles are concentrated in the upper left of the figure, and embryos after the expanded blastocyst stage indicated by squares are concentrated in the lower part of the figure. From such a distribution tendency, the coordinate plane can be divided into six regions (1) to (6) using the threshold values A1, T1, and T2 indicated by dotted lines in Figure 5. The value of the threshold A1 experimentally obtained from the scatter plot is approximately 0.88. Also, the values of the threshold T1 and T2 are approximately 13 μm and 2 μm, respectively.

[0056] Among these regions, region (3) is considered to correspond to the developmental stage before the expanded blastocyst stage because the zona pellucida is thicker than the threshold T1, the aspect ratio is close to 1, and the outer shape of the embryo is generally circular. Regarding region (2), since the outer shape of the embryo is generally circular but the zona pellucida ZP is thinner than the threshold T1, it is considered to correspond to the expanded blastocyst stage.

[0057] Regarding region (5), since the zona pellucida is thin and the aspect ratio is smaller than the threshold A1 and the outer shape of the embryo deviates from a circle, it is considered to correspond to the stage after the expanded blastocyst stage. In the case of a hatched blastocyst, there are also those in which the detected thickness of the zona pellucida ZP becomes substantially zero while having a spherical (circular in the image) outer shape. Regions (1) and (4) where the zona pellucida ZP is extremely thin, that is, smaller than the threshold T2, can be considered to correspond to such a state.

[0058] In region (6), the thickness of the zona pellucida ZP is larger than the threshold T1, and in this regard, it is in a state before the expanded blastocyst stage, but the aspect ratio is small and the outer shape of the embryo greatly deviates from a circle. This state is considered to correspond to an embryo subjected to assisted hatching. Assisted hatching is a treatment that promotes the hatching of a fertilized egg by cutting a part of the zona pellucida using, for example, a laser beam. Region (6) can be considered to correspond to a state in which hatching has started and the outer shape has collapsed due to the zona pellucida being cut before it becomes sufficiently thin.

[0059] In this way, by combining the index value corresponding to the thickness of the zona pellucida ZP and the index value representing the outer shape, the state of the embryo is quantified, and based on the threshold values set for the respective evaluation values, it is possible to objectively and automatically determine in which developmental stage the embryo to be evaluated is. Regarding the threshold values, by determining them based on the determination results of sufficiently trained experts (specifically, embryo culturists), it is possible to obtain determination results that are not inferior to those of experts.

[0060] Here, a method for creating a determination criterion for determining in which developmental stage the embryo is from two index values is described by dividing the feature amount space into a plurality of regions by simple threshold values and associating each region with a developmental stage. In addition to this, it is also possible to create a determination criterion using, for example, known classification techniques. That is, it can be considered that embryos in each developmental stage form clusters for each developmental stage in the two-dimensional feature amount space. Therefore, if the boundaries of the clusters occupied by each developmental stage in the two-dimensional feature amount space are specified using an appropriate learning algorithm, it is possible to perform stage determination based on which cluster the embryo belongs to from the feature amounts obtained for the embryo to be evaluated.

[0061] As described above, the principle of embryo developmental stage determination in this embodiment has been explained. Subsequently, the specific processing content of the image processing of this embodiment incorporating the stage determination based on the above principle will be described. Here, a method using the threshold values shown in FIG. 5 as the determination criterion will be adopted.

[0062] FIG. 6 is a flowchart showing the image processing in this embodiment. As described above, this processing is realized by the CPU 31 executing a control program prepared in advance to cause each part of the apparatus to perform a predetermined operation and periodically imaging the sample S. When the sample container 11 containing the embryo to be evaluated is taken out of the incubator and set in the holding unit 10 (step S101), optical microscope imaging by the microscope imaging unit 28 is performed with the embryo as the imaging object.

[0063] In the processes of steps S102 to S105, two-dimensional image data of the embryo is acquired by imaging the embryo with an optical microscope, and the thickness of the zona pellucida is calculated based on the image data. Specifically, the microscope imaging unit 28 is positioned at an imaging position where the embryo to be evaluated can be placed within the imaging field of view, and the focal position is set to be changed in multiple steps in the depth direction (Z direction), and imaging is performed each time. As a result, a plurality of image sets with different depths of focus, so-called Z-stack images, are acquired (step S102).

[0064] From each of these images, a region corresponding to the zona pellucida ZP is extracted (step S103). Then, out of the plurality of images, one image that is in the best focus with respect to the zona pellucida ZP is selected (step S104), and based on the selected image, the thickness of the zona pellucida ZP is calculated (step S105). The details of these processes, that is, the process of extracting the region corresponding to the zona pellucida ZP (step S103), the process of selecting the most in-focus image (step S104), and the process of calculating the thickness of the zona pellucida ZP (step S105) will be described later. By the processes up to this point, the thickness of the zona pellucida ZP in the embryo E to be evaluated is known. The thickness of the zona pellucida ZP thus obtained is used as the first index value.

[0065] Also, from the image that is in the best focus with respect to the zona pellucida ZP, that is, the image in which the contour of the embryo is the clearest, an overall region corresponding to the entire embryo is extracted (step S106). The extraction of the overall region can be performed by the same image processing as the extraction of the zona pellucida ZP. The aspect ratio of the embryo is calculated from the overall region thus extracted (step S107). Specifically, the image is binarized based on the extraction result of the overall region. Then, a rectangle whose long side, short side, and rotation angle can be changed is fitted to the overall region, and the one with the smallest area among the rectangles circumscribing the periphery of the overall region (circumscribing rectangle) is searched for. The ratio of the short side to the long side of this circumscribing rectangle is used as the aspect ratio, and this is used as the second index value.

[0066] By applying the two thus obtained index values to a predetermined determination criterion, the developmental stage of the embryo is determined (step S108). Specifically, the two index values of the zona pellucida thickness and the aspect ratio are compared with the threshold values for the respective index values obtained in advance based on the above principle, and the developmental stage of the embryo to be evaluated is determined according to which of the regions (1) to (6) the embryo is located in.

[0067] Subsequently, the determination result of the developmental stage of the embryo is compared with the determination result by the previous imaging (step S109). If it is determined that the developmental stage has changed from the previous imaging (YES in step S109), the embryo is OCT-imaged by the imaging unit 20. By doing so, a three-dimensional image of the embryo is acquired every time the developmental stage of the embryo changes. On the other hand, if there is no change in the developmental stage (NO in step S109), the OCT imaging is skipped. By doing so, unnecessary OCT imaging is omitted, and it is possible to prevent the imaging of other embryos from being delayed.

[0068] The imaging of the embryo is periodically performed at a fixed cycle. That is, when the imaging is completed, the sample container 11 is returned to the incubator, taken out again when the next imaging timing arrives, and the processing from step S101 is repeated. When the image processing apparatus 1 and the incubator are integrated such that the holding unit 10 is placed in the culture environment, step S101 can be omitted, and the apparatus waits until the next imaging timing arrives. Thereby, time-lapse observation of the embryo becomes possible.

[0069] In this way, while determining the developmental stage based on the image obtained by optical microscope imaging that enables imaging in a relatively short time, by performing OCT imaging at the necessary timing, in this embodiment, it is possible to perform time-lapse observation at a relatively short cycle, and by performing OCT imaging when it is determined to be necessary, it is also possible to surely acquire the three-dimensional image necessary for the evaluation of the embryo.

[0070] Next, each of the elemental technologies for executing each step (steps S103 to S105, S106) of the above processing will be separately described in order. Note that the processing for acquiring the Z-stack image in step S102 is well-known, so the description thereof will be omitted.

[0071] In step S103, a region corresponding to the zona pellucida ZP is extracted from the two-dimensional image data obtained by imaging the embryo with an optical microscope. This processing can be executed using appropriate image processing techniques. For example, a pattern recognition technique for extracting a region having specific features from within the image can be applied. Specifically, a classification model is constructed by supervised learning using, as a teacher image, an image of the zona pellucida acquired in advance, and by regionally dividing the optical microscope image of the embryo to be evaluated using this model, a region corresponding to the zona pellucida ZP can be extracted from the image.

[0072] In this embodiment, a known semantic segmentation method is used as an example of the region division processing. The semantic segmentation method is a technique for labeling each pixel in an image using a classification model constructed in advance by a deep learning algorithm. In this embodiment, this method can be utilized as follows.

[0073] First, an optical microscope image of an embryo in which the zona pellucida is imaged with good image quality is prepared, and each pixel in the region corresponding to the zona pellucida in the image is labeled to indicate that fact. Then, by executing deep learning with the original microscope image as input data and the label image as correct answer data, a classification model is constructed. By thus giving an unknown image as input data to the classification model constructed in advance, an output image in which a label indicating that fact is attached to the region corresponding to the zona pellucida in the image can be obtained. By extracting such a region from the output image, it is possible to extract the zona pellucida as a result.

[0074] FIG. 7 is a flowchart showing an example of a specific method for constructing a classification model. This process can be executed by various computer devices having a function of displaying an image and a function of receiving an operation input from a user. For example, it can be executed by an image processing apparatus 1 or a general-purpose computer device such as a personal computer.

[0075] First, an optical microscope image of an embryo, which has been previously captured in a state of being in focus on the zona pellucida, is displayed (step S201). In the image processing apparatus 1, the image can be displayed on the display unit 352. For the image thus displayed, a teaching input from the user for designating a region corresponding to the in-focus zona pellucida is received (step S202). In this case, it is desirable that the user is an expert having sufficient knowledge about the image of the embryo. Also, when the image processing apparatus 1 is used, the teaching input can be received via the input device 351.

[0076] A label indicating that is attached to the region designated as the zona pellucida (step S203). By using the image thus labeled as correct data and the original image as input data to execute deep learning, a classification model for extracting the zona pellucida from the image is constructed (step S204). If necessary, labels other than the zona pellucida may be used.

[0077] This classification model is constructed with an image in focus on the zona pellucida as the input image. Therefore, when this is applied to an unknown test image and the semantic segmentation method is executed, a region strongly having the characteristics of the zona pellucida is extracted from the test image. When the zona pellucida is thus extracted, it becomes possible to obtain its thickness. In order to accurately obtain the thickness, it is desirable that the image is in focus on the zona pellucida in as wide an area as possible within the image. That is, it can be said that the calculation of the thickness of the zona pellucida is desirably performed based on an image in which the area of the region extracted by the semantic segmentation method is as wide as possible. Note that the area can be represented by the number of pixels.

[0078] On the other hand, since the embryo to be evaluated has a three-dimensional structure, in the images captured with the depth of focus appropriately determined, the in-focus state with respect to the zona pellucida ZP may not always be good. Therefore, from the Z-stack images obtained by varying the focal position in the depth direction (Z direction), one image with the largest area of the region extracted as corresponding to the zona pellucida ZP is selected, and this image is used for calculating the thickness of the zona pellucida ZP.

[0079] In step S104, the image that is most in focus with respect to the zona pellucida ZP is selected from the Z-stack images. In the semantic segmentation method in this embodiment, since the in-focus region of the zona pellucida is extracted from the image, it can be said that the probability that the image with the largest area of this region is the image that is most in focus with respect to the zona pellucida ZP is high. An image that meets such conditions may be selected.

[0080] However, for example, due to vibrations during imaging or the like, blurring may occur in the image, and as a result, the apparent area of the zona pellucida may appear larger in the image than the actual area. Then, the area extracted as corresponding to the zona pellucida becomes apparently larger, and there is a risk that this position may be misjudged as the in-focus position.

[0081] By using the luminance difference between the pixels sandwiching the peripheral part of the extracted region, it is possible to solve this problem. That is, in a well-focused image, the boundary between the region corresponding to the zona pellucida and its surrounding region is clear, and therefore it is considered that there is also a clear contrast between the luminances of these regions. On the other hand, in an out-of-focus image, the boundary between these regions is not clear, and therefore the contrast is not sharp.

[0082] From this, instead of simply evaluating only based on the size (number of pixels) of the area of the extracted region, it is considered that the above-mentioned misjudgment can be reduced by introducing an evaluation value that also reflects the magnitude of the luminance change at the edge portion of the region, that is, the sharpness. There are various methods for quantifying such an edge change amount, and they can be appropriately selected and applied.

[0083] In this embodiment, as an example, a value obtained by multiplying the area of the extracted region by a coefficient reflecting the magnitude of the luminance change at the edge portion is used as the evaluation value. As this coefficient, for example, a value obtained by squaring the difference in luminance between pixels sandwiching the edge can be used. More specifically, the difference between the average value of the luminances of all the pixels adjacent to the edge among the extracted regions and the average value of the activations of all the pixels adjacent to the edge outside the edge is obtained, and the coefficient is obtained by squaring this difference.

[0084] By doing so, it is possible to reduce the risk that an increase in the extraction area due to the influence of misalignment in the image causes an incorrect determination of the focus position. Here, the luminance difference is squared in order to make the coefficient a positive value, but instead, the absolute value of the luminance difference may be used as the coefficient.

[0085] FIG. 8 is a flowchart showing a process for selecting an image focused on the transparent zone, and the content of this process corresponds to step S104 in FIG. 6. For each image constituting the Z-stack image, a region corresponding to the transparent zone is extracted in step S103. In order to obtain the area of the region thus extracted, the number of pixels belonging to the region is counted for each image (step S301). Then, the average luminance of the pixels adjacent to the inside and outside of the edge of the extracted region is obtained, and the difference is calculated (step S302). Based on these values, an evaluation value indicating the degree of focus of each image is calculated (step S303). Specifically, the evaluation value is calculated by multiplying the number of pixels in the extraction region by a coefficient represented by the square of the luminance difference inside and outside the edge. The image with the largest evaluation value thus obtained is selected as the image most focused on the transparent zone.

[0086] Next, the processing content of step S105 for calculating the thickness of the zona pellucida ZP from one optical microscope image selected from the Z-stack images will be described. For the selected image, in step S103, an area corresponding to the zona pellucida ZP is extracted. In a well-cultured embryo, it is considered that an annular region having a generally constant width is extracted as the region corresponding to the zona pellucida ZP. In an image focused on the zona pellucida ZP, the width of this annulus, that is, the distance between the inner edge and the outer edge of the annulus, corresponds to the thickness of the zona pellucida ZP.

[0087] As methods for obtaining the width of the annulus, various methods can be considered. As a simple method, there is a method using the Distance Transform function equipped in the OpenCV (Open Source Computer Vision) library. By applying the Distance Transform function with one pixel on the inner edge of the annulus as the target pixel, the distance to the pixel on the outer edge of the annulus that is closest to the target pixel can be specified. This distance represents the width of the annulus at that position, that is, the thickness of the zona pellucida Z. Conversely, it is equivalent to taking a pixel on the outer edge of the annulus as the target pixel and obtaining the shortest distance from this to the inner edge. In this way, the average value or median value of the widths obtained at each position on the annulus can be used as an index value that typically represents the thickness of the zona pellucida ZP.

[0088] The detailed processing content of steps S103 to S105 has been described above. On the other hand, step S106 for extracting the entire area of the embryo can basically be performed in the same manner as the process for extracting the zona pellucida ZP. Specifically, in step S202 of FIG. 7, instead of receiving a teaching input for the "area corresponding to the zona pellucida", a teaching input may be received for the "area corresponding to the periphery of the cell". It is desirable that the teacher image in this case also includes an image of an embryo after the expanded blastocyst stage.

[0089] The classification model constructed by performing deep learning using the image thus instructed as correct data has a function of extracting a region corresponding to the whole cell from the input image by a semantic segmentation method. For other processes, the same can be done as in the extraction process of the zona pellucida. By using the learning model thus obtained, it becomes possible to execute the process of step S106 of extracting the whole region from the image.

[0090] Next, a modification example of the image processing of the above embodiment will be described. As shown in FIG. 3, in the embryo at a stage before the expanded blastocyst stage, the variation in the zona pellucida thickness is relatively large. This is considered to be due to individual variations such as differences in the size of the fertilized egg from the beginning of development. Therefore, there is a possibility that the threshold value set for the zona pellucida thickness may vary depending on the collected images.

[0091] In view of this point, in the modification example shown below, instead of using the zona pellucida thickness itself as an index value, the amount of change in the zona pellucida thickness from the initial stage of development is used as an index value, thereby suppressing variations due to individual differences.

[0092] FIG. 9 is a flowchart showing a modification example of image processing. In this modification example, step S121 is added before step S101, and step S122 is added after step S105. Since the processing content except for this point is the same as that in FIG. 6, the same reference numerals are given to the processing steps with the same content, and the description thereof will be omitted.

[0093] First, a reference image is captured, and the thickness of the zona pellucida ZP in the embryo in the reference image is calculated (step S121). The reference image is an image of an embryo at the above-described initial stage of development, and is obtained by optically microscopically imaging an embryo that is known in advance to be at a stage before the expanded blastocyst stage. By using such an embryo as sample S and performing the same processing as in steps S101 to S105, the zona pellucida thickness in the reference image can be obtained.

[0094] In step S122, for the zona pellucida thickness obtained for the embryo with advanced growth, the difference or ratio from the zona pellucida thickness in the reference image is obtained as the "thickness change amount". This thickness change amount represents how much the thickness of the zona pellucida has changed compared to the initial stage. Using this as one of the index values, stage determination can be performed in combination with the aspect ratio in the same way as the above image processing.

[0095] FIG. 10 is a diagram showing the distribution of the thickness change amount of the zona pellucida of the embryo. Comparing with FIG. 3, it can be seen that the variation in values, especially before the expanded blastocyst stage, is suppressed. From this, it is expected that the determination accuracy will be further improved by performing stage determination in combination with the thickness change amount as the first index value and the second index value regarding the outer shape of the embryo.

[0096] As described above, the image processing apparatus 1 of the above embodiment corresponds to an embodiment of the "imaging apparatus" of the present invention. In this embodiment, among the imaging unit 20, the microscope imaging unit 28 functions as the "two-dimensional image acquisition unit" of the present invention, while the other parts function as the "three-dimensional image acquisition unit" of the present invention. Also, the control unit 30, particularly the CPU 31 and the signal processing unit 33, function as the "image processing unit" of the present invention.

[0097] Note that the present invention is not limited to the above-described embodiment, and various modifications can be made other than those described above without departing from the gist thereof. For example, the image processing in the above embodiment includes a process of determining whether to perform OCT imaging based on the result of stage determination and performing OCT imaging if necessary. However, the components of the method for determining the developmental stage of a fertilized egg according to the present invention are the steps from acquiring an optical microscope image to determining the developmental stage of the embryo, and the usage mode of the determination result is not limited to the determination of the necessity of OCT imaging as described above and is arbitrary. Therefore, it is also possible to execute the stage determination method of the present invention using an optical microscope without an OCT imaging function.

[0098] For example, the image processing apparatus 1 of the above embodiment has a function of performing OCT imaging and optical microscope imaging on the sample S, and a function of creating and outputting an output image from the imaging data. However, the generation stage determination method of the present invention does not have an imaging function by itself, and can also be executed by a computer device that has acquired imaging data obtained by imaging with another device having an imaging function. In order to enable this, the present invention may be implemented as a software program for causing a computer device to execute steps other than step S101 among the processing steps in FIG. 6.

[0099] Distribution of such a program can be performed, for example, in a form of downloading via a telecommunication line such as the Internet, and is also possible by distributing a computer-readable recording medium in which the program is non-temporarily recorded. Further, by causing an existing imaging device having an optical microscope imaging function and an OCT imaging function to read this program via an interface, it is also possible to implement the present invention by the device.

[0100] In addition, the above embodiment is configured to execute OCT imaging for an embryo in which there has been a change in the generation stage based on the determination result of the generation stage. That is, the criterion for determining the necessity of OCT imaging is whether or not there has been a change in the generation stage. However, the criterion for determining the necessity of OCT imaging from the result of stage determination is not limited to this.

[0101] As described above by exemplifying specific embodiments, in the generation stage determination method according to the present invention, the determination criterion can be determined based on, for example, index values representing the degree of coincidence and thickness obtained from images of a plurality of fertilized eggs, and information indicating the generation stage of each of the plurality of fertilized eggs. If there are a plurality of images of embryos for which information indicating which generation stage they are in can be used, it is possible to specify the correlation between the index values obtained from those images and the generation stage of the embryo, and the result can be used as a determination criterion in the stage determination of an unknown embryo.

[0102] For example, the determination criteria can be defined by dividing a two-dimensional feature amount space having, as feature amounts, an index value representing the degree of coincidence and an index value representing the thickness into a plurality of regions according to the values of these index values, and associating each region with a development stage. More specifically, a method of setting a threshold value for each of the index value representing the degree of coincidence and the index value representing the thickness, and dividing the two-dimensional feature amount space into a plurality of regions by the threshold value, or a method of dividing the two-dimensional feature amount space into a plurality of regions by the boundary of clusters for each development stage in which images of a plurality of fertilized eggs are formed in the two-dimensional feature amount space is applicable.

[0103] For example, the determination criteria can also be defined by dividing a two-dimensional feature amount space having, as feature amounts, an index value representing the degree of coincidence and an index value representing the change in thickness with respect to a reference image captured in advance into a plurality of regions according to the values of these index values, and associating each region with a development stage. Due to individual differences in fertilized eggs, there may be variations in the thickness of the zona pellucida even among embryos at the same development stage. Therefore, when the thickness of the zona pellucida is directly used as the index value, there is a possibility that variations may occur in the determination results. For example, if the amount of change over time in the thickness of the zona pellucida in each embryo is used as the index value, it becomes possible to suppress the variations caused by such individual differences.

[0104] In these methods, the index value representing the degree of coincidence can be represented, for example, as the ratio of the long side to the short side of the circumscribed rectangle of the fertilized egg in the image. The arithmetic function for searching for the circumscribed rectangle of the image object is standardly equipped in general image processing software. By using this, it becomes possible to easily derive one of the index values serving as the determination criteria for the embryo.

[0105] The method for determining the developmental stage according to the present invention can be used for the purpose of determining whether the developmental stage of a fertilized egg is before the expanded blastocyst stage, the expanded blastocyst stage, or after the expanded blastocyst stage. According to the findings of the inventors of the present application described above, by combining the index value related to the thickness of the zona pellucida and the index value related to the outer shape of the whole embryo, it is possible to effectively distinguish each of these developmental stages.

[0106] For example, when the index value representing the thickness is greater than the threshold value set for the index value, it can be determined that the developmental stage of the fertilized egg is before the expanded blastocyst stage. Also, when the index value representing the thickness is smaller than the threshold value set for the index value and the index value representing the degree of coincidence is greater than the threshold value set for the index value, it can be determined that the developmental stage of the fertilized egg is the expanded blastocyst stage.

[0107] In addition, in the method for determining the developmental stage according to the present invention, the extraction of the region corresponding to the zona pellucida may be performed using a classification algorithm that has been machine-learned in advance with the image of the zona pellucida captured by an optical microscope as a teacher image. According to such a configuration, by using an appropriate classification algorithm, it is possible to extract, with high accuracy, a region having strong morphological characteristics of the zona pellucida from the optical microscope captured image.

[0108] For example, as the classification algorithm, a semantic segmentation method can be used. According to this method, it is possible to divide an image into pixels according to its features. Therefore, the region of the zona pellucida can be accurately extracted from the microscopic image, and its thickness can be appropriately evaluated.

[0109] Also, for example, the optical microscope imaging of the fertilized egg may be performed multiple times with different depths of focus, and the average thickness of the zona pellucida may be obtained from the image in which the area of the region corresponding to the zona pellucida is the largest among those images. The image in which the zona pellucida is extracted in a wide range is likely to be the image in which the zona pellucida is in the most in-focus state, that is, the image captured with the zona pellucida being clear. By using such an image, it is possible to accurately obtain the thickness of the zona pellucida.

Industrial Applicability

[0110] This invention can be applied for the purpose of assisting the work of evaluating the state of cultured embryos, and is suitable for, for example, the purpose of periodically observing embryos and imaging three-dimensional images of embryos at the necessary timing in assisted reproductive medicine.

Explanation of Signs

[0111] 1 Image processing apparatus 10 Holding part 11 Sample container 20 Imaging part (three-dimensional image acquisition part) 28 Microscopic imaging unit (two-dimensional image acquisition part) 30 Control part (image processing part) S Sample

Claims

1. A step of obtaining image data corresponding to an image obtained by optically microscopically imaging a fertilized egg; a step of extracting a region corresponding to the fertilized egg in the image based on the image data, and obtaining an index value representing the degree of coincidence of the region with a circle; a step of extracting a region corresponding to the zona pellucida of the fertilized egg in the image based on the image data, and obtaining an index value representing the thickness of the region; a step of determining the developmental stage of the fertilized egg based on the index value representing the degree of coincidence and the index value representing the thickness, and a predetermined determination criterion for combinations thereof A method for determining the developmental stage of a fertilized egg, comprising:

2. The determination criterion is determined based on the index values representing the degree of coincidence and the thickness respectively obtained from images of a plurality of fertilized eggs, and information indicating the developmental stage of each of the plurality of fertilized eggs. The method for determining the developmental stage of a fertilized egg according to Claim 1.

3. The determination criterion divides a two-dimensional feature amount space having the index value representing the degree of coincidence and the index value representing the thickness as feature amounts into a plurality of regions according to the values of these index values, and associates each region with a developmental stage. The method for determining the developmental stage of a fertilized egg according to Claim 2.

4. A threshold is set for each of the index value representing the degree of coincidence and the index value representing the thickness, and the two-dimensional feature amount space is divided into a plurality of regions by the threshold. The method for determining the developmental stage of a fertilized egg according to Claim 3.

5. The two-dimensional feature amount space is divided into a plurality of regions by the boundaries of clusters for each developmental stage formed by the images of the plurality of fertilized eggs in the two-dimensional feature amount space. The method for determining the developmental stage of a fertilized egg according to Claim 3.

6. The determination criterion divides a two-dimensional feature amount space having the index value representing the degree of coincidence and the index value representing the change in the thickness with respect to a reference image imaged in advance as feature amounts into a plurality of regions according to the values of these index values, and associates each region with a developmental stage. The method for determining the developmental stage of a fertilized egg according to Claim 2.

7. The index value representing the degree of coincidence is represented as the ratio of the long side to the short side of the circumscribed rectangle of the fertilized egg in the image. The method for determining the developmental stage of a fertilized egg according to any one of Claims 1 to 6.

8. The index value representing the thickness is a value corresponding to the amount of change between the thickness of the region corresponding to the zona pellucida and a reference image in which the fertilized egg was imaged in advance and the image, according to the method for determining the developmental stage of a fertilized egg according to any one of claims 1 to 7.

9. The method for determining the developmental stage of a fertilized egg according to any one of claims 1 to 8, which determines whether the developmental stage of the fertilized egg is before the expanded blastocyst stage, the expanded blastocyst stage, or after the expanded blastocyst stage.

10. The method for determining the developmental stage of a fertilized egg according to claim 9, wherein when the index value representing the thickness is greater than a threshold value set for the index value, it is determined that the developmental stage of the fertilized egg is before the expanded blastocyst stage.

11. The method for determining the developmental stage of a fertilized egg according to claim 9 or 10, wherein when the index value representing the thickness is less than a threshold value set for the index value and the index value representing the degree of coincidence is greater than a threshold value set for the index value, it is determined that the developmental stage of the fertilized egg is the expanded blastocyst stage.

12. The extraction of the region corresponding to the zona pellucida is performed using a classification algorithm pre-trained with the image of the zona pellucida captured by optical microscopy as a teacher image, according to the method for determining the developmental stage of a fertilized egg according to any one of claims 1 to 11.

13. The method for determining the developmental stage of a fertilized egg according to claim 12, which uses a semantic segmentation method as the classification algorithm.

14. The optical microscopy imaging of the fertilized egg is performed multiple times with different depths of focus, and the thickness of the zona pellucida is obtained from the image in which the area of the region corresponding to the zona pellucida is the largest among those images, according to the method for determining the developmental stage of a fertilized egg according to claim 12 or 13.

15. A step of determining the developmental stage of the fertilized egg by the method for determining the developmental stage of a fertilized egg according to any one of claims 1 to 14, and a step of performing optical coherence tomography imaging on the fertilized egg according to the result of the determination and an imaging method comprising the steps.

16. A step of obtaining image data corresponding to an image obtained by imaging a fertilized egg with an optical microscope, a step of extracting a region corresponding to the fertilized egg in the image based on the image data and obtaining an index value representing the degree of coincidence of the region with a circle, a step of extracting a region corresponding to the zona pellucida of the fertilized egg in the image based on the image data and obtaining an index value representing the thickness of the region A step of determining the developmental stage of the fertilized egg based on an index value representing the degree of coincidence and an index value representing the thickness, and a determination criterion predetermined for these combinations A program for causing a computer to execute the above steps.

17. A computer-readable recording medium on which the program according to claim 16 is non-temporarily recorded.

18. A two-dimensional image acquisition unit that captures an optical microscope image of a fertilized egg to obtain two-dimensional image data, A three-dimensional image acquisition unit that captures an optical coherence tomography image of the fertilized egg to obtain three-dimensional image data, Based on the two-dimensional image data corresponding to one image of the fertilized egg, an area corresponding to the fertilized egg in the image is extracted, an index value representing the degree of coincidence with a circle of the area is obtained, and based on the two-dimensional image data, an area corresponding to the zona pellucida of the fertilized egg in the image is extracted, an index value representing the thickness of the area is obtained, and based on the index value representing the degree of coincidence and the index value representing the thickness, and a determination criterion predetermined for these combinations, an image processing unit that determines the developmental stage of the fertilized egg Comprising: An imaging device that performs optical coherence tomography imaging of the fertilized egg by the three-dimensional image acquisition unit according to the result of determination by the image processing unit.

Images