Image processing method, fertilized egg evaluation method, computer program, and recording medium

Abstract

To provide a technique capable of acquiring quantitative information being an index in evaluating the state of a fertilized egg from image data obtained by OCT imaging the fertilized egg in the initial stage of generation.SOLUTION: An image processing method according to the present invention comprises: a process of acquiring three-dimensional image data showing a three-dimensional image of a fertilized egg, which is obtained by performing optical coherence tomography imaging to the fertilized egg in a division period; a process of extracting a blastomere region occupied by each blastomere in the three-dimensional image for each blastomere on the basis of three-dimensional image data; a process of obtaining the volume of each blastomere region on the basis of the three-dimensional image data; and a process of obtaining an index value indexing volume variation of the blastomere on the basis of a derived result of the volume. Herein, the index value is obtained on the basis of the maximum value out of differences obtained between the volumes of each blastomere region and their average value or median.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to image processing suitable for imaging a fertilized egg (embryo), particularly a fertilized egg at the cleavage stage, and evaluating its state.

Background Art

[0002] For example, in assisted reproductive medicine for the purpose of infertility treatment, an embryo (fertilized egg) that has been fertilized outside the body and cultured for a certain period is returned to the body. However, the pregnancy success rate (in assisted reproductive medicine) is not necessarily high, and the mental and economic burdens on patients are also great. In order to solve this problem, a method for accurately judging the state of the cultured embryo has been sought.

[0003] For example, as a general evaluation criterion in the initial stage of embryogenesis until the fertilized egg grows into a morula, the Veeck classification is known. This serves as a guideline for classifying the state of a fertilized egg into multiple stages based on the number and size uniformity of blastomeres generated by cleavage and the fragmentation rate.

[0004] Conventionally, for example, by using an apparatus called a time-lapse incubator and performing microscopic imaging of a fertilized egg at regular intervals while placing it in a culture environment, the state of the fertilized egg has been observed and evaluated. However, since optical microscope imaging captures a fertilized egg from one line-of-sight direction, it cannot be said to be sufficient for accurately representing the three-dimensional structure of the fertilized egg. For this reason, regarding the evaluation of a fertilized egg, although there are criteria such as the above-mentioned Veeck classification, the current situation is that it is ultimately left to the subjective judgment of an expert called an embryologist.

[0005] Therefore, there is a need for a technology that can perform such evaluations quantitatively and objectively. As a technology expected to meet this requirement, research has also been conducted on analyzing three-dimensional images of embryos (fertilized eggs) captured by non-invasive tomography techniques such as optical coherence tomography (optical coherence tomography; OCT), and obtaining quantitative information therefrom. For example, Patent Document 1 previously disclosed by the applicant of the present application describes a technique for dividing a three-dimensional image of an embryo (fertilized egg) obtained by OCT imaging into a plurality of regions according to its structure. By using this technology, it is expected that, for example, individual blastomeres and fragmentations can be separately extracted from a fertilized egg (embryo) at the cleavage stage.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] However, as described above, heretofore, evaluation techniques based on qualitative and subjective evaluation criteria have been predominantly put into practical use. Therefore, it cannot be said that the criteria for quantitatively and objectively evaluating fertilized eggs have been sufficiently established. That is, when capturing a fertilized egg in its three-dimensional structure, converting it into image data, and analyzing it as in the above-described technology, there is currently no clear indicator as to what quantitative information should be obtained and how it should be utilized.

[0008] This invention has been made in view of the above problems, and an object thereof is to provide a technology capable of obtaining quantitative information serving as an indicator for evaluating the state of a fertilized egg from image data obtained by OCT imaging of a fertilized egg in the early stage of development.

Means for Solving the Problems

[0009] One aspect of this invention includes a step of acquiring three-dimensional image data representing a three-dimensional image of a fertilized egg obtained by performing optical coherence tomography imaging on the fertilized egg at the cleavage stage, a step of extracting, for each blastomere, a blastomere region occupied by the individual blastomere in the three-dimensional image based on the three-dimensional image data, a step of obtaining the volume of each of the blastomere regions based on the three-dimensional image data, and a step of obtaining an index value that indicates the volume variation of the blastomeres based on the derivation result of the volume. This is an image processing method.

[0010] Another aspect of this invention is an evaluation method for a fertilized egg, which includes a step of performing optical coherence tomography imaging on a fertilized egg at the cleavage stage to obtain three-dimensional image data representing a three-dimensional image of the fertilized egg, a step of extracting, for each blastomere, a blastomere region occupied by the individual blastomere in the three-dimensional image based on the three-dimensional image data, a step of obtaining the volume of each of the blastomere regions based on the three-dimensional image data, and a step of obtaining an index value that indicates the volume variation of the blastomeres based on the derivation result of the volume.

[0011] In these inventions, the index value is obtained based on the maximum value among the differences obtained between the volume of each individual blastomere region and their average value or median value.

[0012] In the invention configured as described above, the volume of each individual blastomere extracted from the three-dimensional image data obtained by OCT imaging is obtained, and an index value representing the degree of variation thereof is obtained. By using the technique described in Patent Document 1 mentioned above, it is possible to identify the regions of the individual blastomeres contained in the fertilized egg.

[0013] Here, in the initial stage of embryo development, it is regarded as one of the conditions for good embryos that the sizes of blastomeres produced by cleavage are uniform. Conventionally, the uniformity has been evaluated mainly by visual judgment by experts, but by using OCT images, more quantitative and accurate measurement based on three-dimensional structure has become possible. Specifically, it is possible to individually obtain the volume of each blastomere from the size of the spatial region (herein referred to as the "blastomere region") occupied by each blastomere identified in the three-dimensional image. And from the obtained volumes, an index value indicating the variation in volume among blastomeres can be obtained. Thereby, it becomes possible to establish a more quantitative and objective evaluation technique in place of the conventional subjective judgment.

[0014] In the present invention, the index value is obtained based on the maximum value among the differences obtained between the volume of each individual blastomere region and the average value or median value thereof. That is, in the present invention, the index value is obtained by focusing on the one having the volume with the largest deviation from the average value or median value among the individual blastomere regions. In the process of cleavage, a case may occur where only a very small number of blastomeres are clearly larger (or smaller) than other blastomeres. In such a case, even if the uniformity of other blastomeres is good, it is not necessarily the case for the whole fertilized egg. However, the evaluation criteria in such a case have not been established yet at present.

[0015] When some blastomeres have a size deviated from other blastomeres as described above, it is expected that the quantitative and objective evaluation of the fertilized egg will proceed more efficiently by providing an index value showing that fact and more preferably quantitatively showing the degree of deviation. The index value of the present invention is based on the maximum value among the degrees of deviation from the standard blastomere size represented by the average value or median value of the volume, which is obtained for each blastomere, and can accurately meet such requirements.

Advantages of the Invention

[0016] As described above, according to the present invention, an index value that quantitatively and objectively represents the uniformity of the blastomere size in an initial fertilized egg can be obtained. Therefore, it is possible to provide quantitative information useful for evaluating the state of the fertilized egg.

Brief Description of the Drawings

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0018] FIG. 1 is a diagram showing a configuration example of an image processing apparatus suitable as an execution entity of the image processing method according to the present invention. This image processing apparatus 1 tomographically images a sample carried in a liquid, for example, a fertilized egg (embryo) cultured in a culture solution, and processes the obtained tomographic image to create a cross-sectional image showing the structure of one cross-section of the sample. Further, a three-dimensional image of the sample is created from a plurality of cross-sectional images. In order to uniformly indicate the directions in the following figures, an XYZ orthogonal coordinate axis is set as shown in FIG. 1. Here, the XY plane represents the horizontal plane. Also, the Z-axis represents the vertical axis, and more specifically, the (-Z) direction represents the vertically downward direction.

[0019] The image processing apparatus 1 includes a holding unit 10. The holding unit 10 holds the sample container 11 that houses the sample S to be imaged in a horizontal posture. The sample container 11 is, for example, a flat-bottomed and shallow dish-shaped container called a dish, having an opening on the upper surface and a recess capable of carrying a liquid. A medium M such as a culture solution is injected into the sample container 11, and a fertilized egg as the sample S is carried inside. In this example, a plurality of samples S are carried in the dish as the sample container 11, but it is not limited to this.

[0020] In this example, a plurality of samples S are carried in the sample container 11 having a single recess, but it is not limited to this. For example, the sample container 11 may be a well plate in which a plurality of recesses called wells are arranged in a single plate-shaped member. In this case, a plurality of samples S can be carried in the plurality of wells one by one. Also, for example, a plurality of dishes each carrying a sample S may be held by the holding unit 10 in a state of being arranged in the horizontal direction and used for imaging.

[0021] Below the sample container 11 held by the holding unit 10, the imaging unit 20 is arranged. An OCT (optical coherence tomography, or optical coherence tomography) device capable of non-contact, non-destructive (non-invasive) imaging of a tomographic image of an object to be imaged is used for the imaging unit 20. Although details will be described later, the imaging unit 20 which is an OCT device includes a light source 21 that generates illumination light for the object to be imaged, an optical fiber coupler 22, an object optical system 23, a reference optical system 24, a spectroscope 25, and a photodetector 26.

[0022] In addition, the image processing device 1 further includes a control unit 30 that controls the operation of the device, and a drive unit 40 that drives the movable part of the imaging unit 20. The control unit 30 includes a CPU (Central Processing Unit) 31, an A / D converter 32, a signal processing unit 33, a 3D restoration unit 34, an interface (IF) unit 35, an image memory 36, and a memory 37.

[0023] The CPU 31 controls the operation of the entire device by executing a predetermined control program, and the control program executed by the CPU 31 and the data generated during processing are stored in the memory 37. The A / D converter 32 converts a signal output according to the amount of received light from the photodetector 26 of the imaging unit 20 into digital data. The signal processing unit 33 performs signal processing described later based on the digital data output from the A / D converter 32 to create a tomographic image of the object to be imaged. The 3D restoration unit 34 has a function of creating a three-dimensional image (3D image) of the imaged cell mass based on the image data of a plurality of tomographic images taken. The image data of the tomographic image created by the signal processing unit 33 and the image data of the three-dimensional image created by the 3D restoration unit 34 are appropriately stored and saved by the image memory 36.

[0024] The interface unit 35 is responsible for the communication between the image processing apparatus 1 and the outside. Specifically, the interface unit 35 has a communication function for communicating with external devices and a user interface function for receiving operation inputs from the user and notifying the user of various types of information. For this purpose, the interface unit 35 is connected to an input device 351 such as a keyboard, a mouse, or a touch panel that can receive operation inputs regarding function selection of the apparatus, setting of operation conditions, etc., and a display unit 352 such as a liquid crystal display that displays various processing results such as tomographic images created by the signal processing unit 33 and three-dimensional images created by the 3D restoration unit 34.

[0025] In the imaging unit 20, a low-coherence light beam including broadband wavelength components is emitted from a light source 21 having a light-emitting element such as a light-emitting diode or a superluminescent diode (SLD). For the purpose of imaging a sample such as cells, it is preferable to use, for example, near-infrared light in order to allow the incident light to reach the inside of the sample.

[0026] The light source 21 is connected to an optical fiber 221 that is one of the optical fibers constituting the optical fiber coupler 22, and the low-coherence light emitted from the light source 21 is branched by the optical fiber coupler 22 into light to two optical fibers 222 and 224. The optical fiber 222 constitutes an object system optical path. More specifically, the light emitted from the end of the optical fiber 222 enters the object optical system 23.

[0027] The object optical system 23 includes a collimator lens 231 and an objective lens 232. The light emitted from the end of the optical fiber 222 enters the objective lens 232 through the collimator lens 231. The objective lens 232 has a function of converging the light (observation light) from the light source 21 as a light beam onto the sample, and a function of condensing the reflected light emitted from the sample and directing it toward the optical fiber coupler 22. Although a single objective lens 232 is shown in the figure, a plurality of optical elements may be combined. The reflected light from the object to be imaged enters the optical fiber 222 as signal light through the objective lens 232 and the collimator lens 231. The optical axis of the objective lens 232 is orthogonal to the bottom surface 111 of the sample container 11, and in this example, the optical axis direction coincides with the vertical axis direction.

[0028] The drive unit 40 is controlled by the CPU 31. That is, the CPU 31 gives a control command to the drive unit 40, and in response, the drive unit 40 causes the imaging unit 20 to move in a predetermined direction. More specifically, the drive unit 40 moves the imaging unit 20 in the horizontal direction (XY direction) and the vertical direction (Z direction). By moving the imaging unit 20 in the horizontal direction, the imaging range changes in the horizontal direction. Also, by moving the imaging unit 20 in the vertical direction, the focal position in the optical axis direction of the objective lens 232 changes with respect to the sample S which is the object to be imaged.

[0029] A part of the light incident from the light source 21 on the optical fiber coupler 22 enters the reference optical system 24 through the optical fiber 224. The reference optical system 24 includes a collimator lens 241 and a reference mirror 243, and these together with the optical fiber 224 constitute a reference optical path. Specifically, the light emitted from the end of the optical fiber 224 enters the reference mirror 243 through the collimator lens 241. The light reflected by the reference mirror 243 enters the optical fiber 224 as reference light.

[0030] The reflected light (signal light) reflected by the surface or internal reflection surface of the sample and the reference light reflected by the reference mirror 243 are mixed by the optical fiber coupler 22 and enter the photodetector 26 through the optical fiber 226. At this time, interference due to the phase difference occurs between the signal light and the reference light, and the spectral spectrum of the interference light varies depending on the depth of the reflection surface. That is, the spectral spectrum of the interference light has information in the depth direction of the object to be imaged. Therefore, by spectroscopically analyzing the interference light for each wavelength, detecting the light quantity, and performing a Fourier transform on the detected interference signal, the reflected light intensity distribution in the depth direction of the object to be imaged can be obtained. The OCT imaging technology based on such a principle is called Fourier Domain OCT (FD-OCT).

[0031] In the imaging unit 20 of this embodiment, a spectroscope 25 is provided on the optical path of the interference light from the optical fiber 226 to the photodetector 26. As the spectroscope 25, for example, one using a prism, one using a diffraction grating, or the like can be used. The interference light is spectroscopically analyzed for each wavelength component by the spectroscope 25 and received by the photodetector 26.

[0032] By performing a Fourier transform on the interference signal output from the photodetector 26 in response to the interference light detected by the photodetector 26, the reflected light intensity distribution in the depth direction, that is, the Z direction, at the incident position of the illumination light in the sample is obtained. By scanning the light beam incident on the sample container 11 in the X direction, the reflected light intensity distribution in a plane parallel to the XZ plane is obtained, and a tomographic image of the sample S with that plane as a cross section can be created. Since the principle is well-known, detailed description is omitted.

[0033] Also, by changing the beam incident position in the Y direction in multiple steps and capturing a tomographic image each time, a large number of tomographic images can be obtained by tomographically imaging the sample in a cross-section parallel to the XZ plane. If the scanning pitch in the Y direction is reduced, image data with sufficient resolution to grasp the three-dimensional structure of the sample can be obtained. From these tomographic image data, three-dimensional image data (so-called voxel data) corresponding to the three-dimensional image of the sample can be created. Thus, the image processing apparatus 1 has a function of creating a tomographic image in an arbitrary cross-section parallel to the XZ plane of the sample S, which is the object to be imaged, and a function of creating a three-dimensional image of the sample S from a plurality of different tomographic images.

[0034] In the present embodiment, the image processing apparatus 1 configured as described above is used for imaging a fertilized egg (embryo) and its quantitative evaluation. Specifically, an OCT image of a fertilized egg at the cleavage stage is captured to obtain its three-dimensional image, and by analyzing this through image processing, an index value indicating the degree of variation in the size of blastomeres constituting the fertilized egg is calculated. Hereinafter, the specific content of this process will be described.

[0035] FIG. 2 is a diagram schematically showing the structure of a fertilized egg (embryo) serving as a sample in the present embodiment. As is already known, when an egg is fertilized, cleavage begins, and a blastocyst is formed through a state called a morula. The image processing method of the present embodiment is suitable for analyzing and evaluating a fertilized egg at the cleavage stage from immediately after fertilization to the morula stage, for example.

[0036] FIG. 2(a) schematically shows the structure of a fertilized egg at an initial stage (for example, from the 4-cell stage to the morula stage). The outer shape of the fertilized egg (embryo) E is generally spherical. Its surface is covered with a jelly-like glycoprotein layer Z called the zona pellucida, and a plurality of cells C generated by cell division of the fertilized egg are contained inside. At this time, the cells are also particularly referred to as "blastomeres".

[0037] As cleavage progresses, the number of blastomeres C increases. In a state where the culture progresses well, the interior of the zona pellucida Z is relatively large and occupied by a plurality of blastomeres C of uniform size. In other words, in a fertilized egg at the cleavage stage, whether the sizes of the blastomeres C constituting the fertilized egg are uniform is one of the indicators for determining the quality of the culture state.

[0038] Using the variation in the size of blastomeres as an indicator for evaluating fertilized eggs is also done in the Veeck classification. However, as described above, the evaluation of the degree of variation is left to the visual judgment of an observer. This embodiment automatically calculates an index value that quantitatively represents the degree of this variation by analyzing an OCT - imaged image.

[0039] As will be described in detail later, for the above - mentioned purpose, the image processing of this embodiment includes a process of extracting the regions occupied by individual blastomeres C from the three - dimensional image of the fertilized egg E obtained by OCT imaging. As image processing for this purpose, for example, the technology described in Patent Document 1 can be preferably applied. Patent Document 1 discloses a technology for performing image processing on a three - dimensional image of a fertilized egg, separating and extracting regions corresponding to the zona pellucida Z, blastomeres C, and minute objects called fragments, and further individually dividing the regions corresponding to individual blastomeres C. Local Thickness calculation and Watershed algorithm are used for region division.

[0040] Figure 2(b) is a diagram showing an example of blastomeres extracted from an OCT image. More specifically, this figure is obtained by applying the image processing method described in Patent Document 1 to the OCT image of a fertilized egg at the cleavage stage. Here, the region corresponding to the zona pellucida extracted from the three - dimensional image of the fertilized egg is erased, and the regions corresponding to individual blastomeres C are represented with different luminances. Although a plurality of blastomeres C are integrated in a state of being in contact with other blastomeres, they can be separated by image processing. This method can also be applied to the extraction of blastomeres C in this embodiment. Therefore, in the following description, the detailed description of the blastomere extraction process will be omitted.

[0041] Figure 3 is a flowchart showing the image processing in this embodiment. This processing is realized by the CPU 31 of the control unit 30 executing a control program prepared in advance to cause each part of the apparatus to perform a predetermined operation. First, three-dimensional image data corresponding to the three-dimensional image of the fertilized egg in the division period to be evaluated is acquired (step S101). Regarding the image data, it may be acquired by newly performing OCT imaging with the fertilized egg as the sample S, or may be in a mode of reading out the image data acquired by past imaging from the image memory 36. Further, it may be in a mode of acquiring the image data stored in an external storage device or the like through a telecommunication line.

[0042] Based on the three-dimensional image data of the fertilized egg thus obtained, a blastomere region corresponding to each blastomere in the three-dimensional image of the fertilized egg is extracted (step S102). More specifically, the image processing described in Patent Document 1 is applied to the three-dimensional image data to divide the three-dimensional image into regions corresponding to the zona pellucida, blastomeres (and, if necessary, fragments) respectively. Regarding the blastomeres, the region is divided on a per-cell basis to identify the blastomere regions corresponding to individual blastomeres. Then, by eliminating objects other than the objects corresponding to the blastomere regions, the blastomere regions corresponding to each blastomere are extracted.

[0043] When the regions corresponding to individual blastomeres in the three-dimensional image are thus identified, the number and volume of the blastomeres are measured (step S103). Hereinafter, the number of blastomeres thus obtained is represented by the symbol X. The volume of each blastomere can be represented, for example, by the number of voxels constituting the blastomere region. Also, the volume of the blastomere in real space can be obtained by multiplying the volume per voxel in real space by the number of voxels.

[0044] Here, it is determined whether the counted number X of blastomeres corresponds to a power of 2 (step S104). If this is the case (YES in step S104), subsequently, blastomere number adjustment processing (step S105) is executed. On the other hand, if the blastomere number X does not correspond to a power of 2 (NO in step S104), step S105 is skipped. The blastomere number adjustment processing is a process for suppressing misevaluation caused by variations in the timing of cell division during the process of cleavage in the evaluation of blastomere variation described later. The content of this processing will be described below.

[0045] FIG. 4 is a diagram for explaining the principle of the blastomere number adjustment processing. In the "Pattern" column of the figure, a circle represents one blastomere. As shown as case (a) in FIG. 4, generally, cleavage proceeds by repeating the cycle in which one cell divides into two for each generation, and the number of blastomeres increases twofold for each generation as 1, 2, 4,.... That is, the number of blastomeres X for each generation can be represented by a geometric sequence with 1 as the first term and 2 as the common ratio.

[0046] However, the division of each cell does not occur strictly simultaneously. For this reason, in an image captured as a still image at a certain time, there may be a case where cells before division and cells after division are mixed. In this case, the blastomere number X may not follow the geometric sequence described above. Also in this embodiment, the blastomere number X obtained in step S103 may not follow the regularity represented by the above geometric sequence.

[0047] Even if the blastomere number deviates from the regularity in this way, if it is caused by the relationship between the timing of cell division and the timing of imaging, this does not immediately indicate that there is a problem with the quality of the fertilized egg. However, for example, when trying to evaluate the state of the fertilized egg based on variations in the size of blastomeres, it may cause misevaluation.

[0048] That is, in a fertilized egg, it is considered a good state that the sizes of blastomeres are uniform, that is, the variation in size is small. Here, generally, a blastomere before division is larger than a blastomere after division. Therefore, when blastomeres before division and blastomeres after division are mixed as described above, the variation obtained between them will naturally be large. However, since this variation is temporarily caused by differences in the timing of cell division, if this result is directly used for evaluation, the evaluation result will be inappropriate.

[0049] Therefore, in this embodiment, when the number of blastomeres X obtained as a result of region division deviates from the above regularity, it is assumed that blastomeres before division and blastomeres after division are mixed. And from the general knowledge that a blastomere before division is larger than a blastomere after division, it is presumed that among the blastomeres, those with a large volume are blastomeres before division. When such a blastomere divides into two, each blastomere becomes approximately half the size of the original blastomere.

[0050] Therefore, it is considered possible to approximately represent the size of each blastomere after division by dividing the original blastomere region so as to bisect its volume. By dividing one blastomere region presumed to be before division into two pseudo blastomere regions in this way, it becomes possible to treat them on the same level as other blastomeres after division, that is, to evaluate each as a cell of the same generation. The blastomere regions created by the division do not necessarily represent the actual shape after division, but are suitably available for the purpose of deriving quantitative information such as size, for example.

[0051] As described above, the number of blastomeres X obtained from one static image can take various values depending on the imaging timing. For example, as shown as case (b) in FIG. 4, as a case where the number of blastomeres X is 3, as shown by the dashed line in the figure, a case consisting of two blastomeres after division from a state of two blastomeres and one blastomere before division can be considered. Also, as case (c) where the number of blastomeres X is 5, a case consisting of two blastomeres after division from a state of four blastomeres and three blastomeres before division can be considered.

[0052] Similarly, for case (d) where the number of blastomeres X is 6, it can be considered as a combination of the four blastomeres formed by the splitting of two blastomeres from the state of four blastomeres and the two blastomeres before splitting. For case (e) where the number of blastomeres X is 7, it can be considered as a combination of the six blastomeres formed by the splitting of three blastomeres from the state of four blastomeres and the one blastomere before splitting. More generally, when the number of blastomeres X is expressed by the following formula using the natural number N: 2 N < X < 2 N+1 … (Equation 1) Among them, (2 N+1 - X) are estimated to be the blastomeres before splitting.

[0053] Therefore, by dividing each of the (2 N+1 - X) blastomeres estimated to be before splitting into two, and rounding the apparent number of blastomeres X to any number (4, 8, …) on the geometric sequence, it becomes possible to reduce the risk of misevaluation in the quantitative evaluation of the fertilized egg. The process for this is the blastomere number adjustment process (step S105).

[0054] In addition, as cases where the number of blastomeres does not follow the regularity of the geometric sequence like this, in addition to the case where blastomeres before and after splitting are mixed as described above, for example, there may also be cases due to abnormalities in the cleavage process such as inactivation of some blastomeres. In such cases, the blastomere number adjustment process as described above does not necessarily return appropriate results. However, in such cases, large variations will also appear in the process of evaluating the variation of blastomeres described later, so it can be avoided being misevaluated as a fertilized egg in good condition.

[0055] Fig. 5 is a diagram showing the blastomere number adjustment process. Fig. 5(a) is a flowchart showing the content of the blastomere number adjustment process. When the extracted number of blastomeres X cannot be expressed by 2 N Based on the above principle, among the blastomere regions, starting from the ones with large volume, (2 N+1(-X) are selected (step S201). These blastomere regions are presumed to correspond to the blastomeres before division. Therefore, these blastomere regions are bisected. Various methods can be considered as the method for bisecting the blastomere regions.

[0056] For example, there is a method of bisecting by dividing the blastomere region with a plane orthogonal to its major axis. Specifically, the axis corresponding to the major axis of each blastomere region is specified (step S202), and a cutting plane for dividing the blastomere region is temporarily set (step S203). The cutting plane is a plane perpendicular to the major axis. The volume ratio of the two regions obtained by dividing the blastomere region with this cutting plane is obtained (step S204), and it is determined whether the value is within the allowable range (step S205).

[0057] For the purpose of bisecting one blastomere region, it is ideal that the volume ratio of the two divided blastomere regions is 1:1. However, in reality, it is sufficient that the two blastomere regions are approximately equal. Based on 1:1, a certain width is given to the above ratio as the allowable range, and it is only necessary to determine whether the volume ratio falls within this allowable range.

[0058] If the volume ratio is within the allowable range (YES in step S204), the region division result at the cutting plane set at that time can be adopted. On the other hand, if the volume ratio is outside the allowable range (NO in step S205), the setting of the cutting plane is redone, and steps S203 to S205 are repeated until the volume ratio falls within the allowable range. Thereby, one blastomere region can be divided into two blastomere regions with approximately equal volumes. The above division is performed for all the blastomere regions selected in step S201. As a result, the number of blastomeres X after division can be represented by 2 N using the natural number N.

[0059] Figure 5(b) shows the number of blastomeres X obtained in step S103 and the number of blastomeres to be divided at that time (2 N+1It is a diagram showing the relationship with (-X). For example, when the number of divided spheres X obtained in step S103 is 3, by bisecting one divided sphere with the largest volume, the final number of divided spheres becomes 4. Also, when the number of divided spheres X is 5, three of the ones with the largest volume are each divided into two, and the final number of divided spheres becomes 8. The same can be considered for any case where the number of divided spheres X is 6 or more. Thus, by dividing some of the divided spheres according to the value of the number of divided spheres X, the final number of divided spheres can be rounded to 2 N pieces. This can avoid the variation in size caused by the deviation in timing between cell division and imaging from becoming a cause of misevaluation.

[0060] Figure 6 is a diagram showing an example of a method for bisecting a divided sphere. In the method described above, as shown in Figure 6(a), for the divided sphere B1 selected in step S201, a cutting plane Sc orthogonal to its major axis Lx is temporarily set. Then, by moving the cutting plane Sc along the major axis Lx so that the two regions divided by this cutting plane Sc have approximately the same volume, the divided sphere region B1 can be divided into two regions having substantially equal volumes.

[0061] In addition to the above, for example, as shown in Figure 6(b), for an irregular divided sphere region B2, by performing a further region division process, for example, a division process using the Watershed algorithm, the divided sphere region B2 can be divided into two regions based on its shape. However, for the purpose of quantitatively representing the variation in the size of the divided spheres in this embodiment, the main focus is on making the volumes of the divided regions approximately the same rather than division based on shape. Therefore, even in such a case, if the major axis Lx is set for the divided sphere region B2 and the cutting plane Sc set by the Watershed algorithm is moved along the major axis Lx as needed when the volumes of the two divided regions are significantly different, it is possible to make the volume ratio of the two regions approach 1:1.

[0062] Returning to FIG. 3, the description of the image processing in this embodiment will be continued. By adjusting the number of blastomeres as needed as described above, at the time of step S106, the blastomeres before division are already treated as two divided blastomeres, and the number of blastomeres X is 2 N and is the numerical value represented thereby. An index value that quantitatively represents the variation in their size, specifically the volume, is obtained among these blastomeres (step S106).

[0063] As described above, it is considered that a good state is represented by the fact that the sizes of the blastomeres in the fertilized egg at the cleavage stage are well aligned. Until now, trained users such as embryologists have visually evaluated the degree of variation in the size of the blastomeres. Therefore, it is expected that if an index value that quantitatively represents the degree of variation in the size of the blastomeres is automatically calculated and presented from the image of the fertilized egg, the workload of the user evaluating the state of the fertilized egg can be greatly reduced.

[0064] Therefore, as a usage mode of the obtained index value, for example, it is conceivable to display and output the index value to the display unit 352 and present it to the user (step S107), thereby assisting the user in evaluating the fertilized egg. Also, as will be described later, a usage mode of automatically determining the quality of the state of the fertilized egg by comparing the obtained index value with an appropriate threshold value is also conceivable.

[0065] As a method for quantifying the variation in volume obtained for each blastomere region, it is conceivable to statistically process those volume values and quantitatively represent them, for example, by the value of variance. Although such a method may be applied, when the population is small as in this case, it is also possible to quantify the variation by, for example, the following method.

[0066] FIG. 7 is a flowchart showing the process of obtaining the variation index value in this embodiment. X (= 2) extracted by the process up to step S105 (FIG. 3) N) The volume values of each of the split balls are sorted along with their sizes, for example, in ascending order (step S301). Also, the average value of their volumes is calculated (step S302). Then, the volume of each split ball is normalized by the obtained average volume value (step S303), and by applying the normalized volume value to a predetermined calculation formula, an index value that quantitatively represents the volume variation of the split balls is calculated (step S304).

[0067] As long as the index value is such that the numerical value increases as the volume variation between the split balls increases, various calculation formulas can be considered for the calculation formula. From the perspective that it is easy for the user to intuitively grasp the degree of variation, for example, when there is no variation, the index value becomes 0 or 1, and it is desirable that the index value increases as the variation increases. Hereinafter, as specific examples of the calculation method of such an index value, three calculation methods are proposed.

[0068] FIG. 8 is a diagram showing three examples of the calculation method of the index value. Here, as a simple example for easy understanding, the case where the number of split balls X after the split ball number adjustment process is 4 is taken. Also, in the following description, each of the four split balls is distinguished by numbers 1, 2, 3, and 4 when sorted in ascending order of volume, and their respective volumes are represented by symbols V1, V2, V3, and V4. Also, the average value of these volumes is represented by the symbol Vm. In each figure, the volume of each split ball is represented as the normalized volume value after being normalized by the volume average value Vm.

[0069] In the first example shown in FIG. 8(a), the growth rate of the volume with respect to the split ball number when the split balls are arranged in ascending order of volume, that is, the slope of the straight line drawn in FIG. 8(a), is used as the index value. For example, using the minimum value (V1 / Vm) and the maximum value (V4 / Vm) of the volume after normalization, the index value Vi is given by the following formula: Vi=(V4 / Vm-V1 / Vm) / (4-1) … (Equation 2) It can be represented by. When there is no variation in the volume between the blastomeres, since the volumes V1 and V4 are equal, the index value Vi represented by (Equation 2) becomes zero. On the other hand, the greater the variation, the greater the difference between the volumes V1 and V4, and the greater the index value Vi represented by (Equation 2). Incidentally, when generalizing (Equation 2), for the number of blastomeres X, the following formula: Vi = (Vx / Vm - V1 / Vm) / (X - 1) … (Equation 3) is obtained.

[0070] Here, the meaning of normalizing the volume of each blastomere by its average value is to eliminate the influence of the size of each blastomere caused by the progress of cleavage or the individual differences of the fertilized eggs. For example, as cleavage progresses, each blastomere becomes smaller, so it is effective to use the normalized volume in order to prevent the index value of variation from being affected by the absolute value of the volume. Here, although the volumes V1 to Vx of each blastomere are normalized by the volume average value Vm and then substituted into the calculation formula, even if the original volumes V1 to Vx are directly substituted into the calculation formula and the calculated value is normalized by the volume average value Vm, the technical meaning remains the same. These points are the same in the following cases.

[0071] In the second case shown in FIG. 8(b), the index value Vi is obtained based on the cumulative value of the normalized volume of each blastomere. Specifically, for the blastomere numbered 2, the cumulative value of the volumes of the two smallest-volume blastomeres 1 and 2 is plotted. Similarly, for the blastomere numbered 3, the cumulative value of the volumes of the three blastomeres 1 to 3 in ascending order of volume is plotted. Furthermore, for the largest blastomere 4, the cumulative value of the volumes of all blastomeres is plotted. The slope of the straight line in the graph thus created is taken as the index value Vi. Generalizing this, the index value Vi is given by the following formula:

Number

[0072] In the third case shown in Fig. 8(c), the index value Vi is obtained based on the variation ratio from the average value of the normalized volume of each divided sphere. Specifically, for each divided sphere, the difference between the normalized volume value and the normalized volume average value (i.e., 1) is obtained, and the value obtained by dividing the sum of their absolute values by the number of divided spheres is defined as the index value Vi. Generally speaking, the index value Vi is expressed by the following formula: [Number] which is represented by

[0073] Regardless of which of these calculation methods is used, the index value Vi is minimized when there is no volume variation between the divided spheres, and the value increases as the variation becomes larger. That is, the magnitude of the index value Vi can quantitatively represent the variation in size between the divided spheres. Instead of normalizing by the average value of the volume, it is also conceivable to normalize by its median, minimum value, maximum value, etc. In particular, when the volume of a certain divided sphere is significantly larger (or smaller) than others, the average value may be significantly biased by its influence. For example, this problem can be avoided by using the median value.

[0074] In addition, in order to handle cases where some divided spheres are significantly larger (or smaller) than others in this way, the following method for calculating the index value can also be applied. Here, the case where the number of divided spheres X after the divided sphere number adjustment process is 4 is taken as an example, and its specific content will be described. Note that the processing other than the calculation of the index value can be the same as that in the above embodiment.

[0075] Fig. 9 is a diagram showing an example of the volume variation of the divided spheres. In the example shown in Fig. 9(a), the normalized volumes of the divided spheres represented by the divided sphere numbers 1 to 4 are 0.9, 0.9, 1.1, and 1.1, respectively. At this time, for example, when the index value Vi is obtained by (Equation 5), Vi = 0.1 is obtained.

[0076] On the other hand, in the case shown in Fig. 9(b), the normalized volumes of the respective divided spheres represented by the divided sphere numbers 1 to 4 are 0.9, 0.9, 1.0, and 1.2, respectively, and only the volume of the divided sphere represented by number 4 is significantly larger than the others. Further, in the case shown in Fig. 9(c), the normalized volumes of the respective divided spheres represented by the divided sphere numbers 1 to 4 are 0.8, 1.0, 1.1, and 1.1, respectively, and only the volume of the divided sphere represented by number 1 is significantly smaller than the others. In these cases as well, the index value Vi obtained by (Equation 5) is 0.1 in each case.

[0077] Thus, there can be a case where no significant difference appears in the index value Vi obtained by a certain calculation method in a plurality of cases that are clearly different in terms of the uniformity of the divided spheres. To address this problem, the following equation:

Equation

[0078] That is, the index value Vi according to this definition is obtained as the maximum value among the values obtained by normalizing the difference between the volume obtained for each divided sphere and the average volume by the average volume. According to this definition, the index value Vi in the case of Fig. 9(a) is 0.1, while the index value Vi in the cases of Fig. 9(b) and Fig. 9(c) is 0.2 in each case, which is larger than the value in the case of Fig. 9(a).

[0079] According to this definition, the index value Vi becomes smaller as the variation in the blastomere volume becomes smaller. On the other hand, the value becomes larger when there is a blastomere that is significantly larger than the others. Similarly, the index value Vi also becomes larger when there is a blastomere that is significantly smaller than the others, and if the degree of deviation from the average value is the same, the numerical values are also approximately equal. This indicates that when some blastomeres are significantly larger (or smaller) than others, the index value Vi obtained by (Equation 6) quantitatively represents the degree of such deviation. It is also possible to quantitatively represent the uniformity of the blastomere volume by such an index value.

[0080] The index value Vi obtained by these calculation methods can be used as a reference when the user determines the state of the fertilized egg. Also, as shown below, it is also possible to automatically determine the state of the fertilized egg from the index value.

[0081] Figure 10 is a flowchart showing an example of a method for evaluating a fertilized egg based on an index value. This evaluation method can be realized by adding several processing steps after the image processing shown in Figure 3. Therefore, the illustration and description of steps S101 to S106, for which the same processing can be adopted, are omitted. Similar to the above processing, in step S107, the calculated index value Vi is displayed and output. Then, the index value Vi is compared with a threshold value Vth preset for the index value (step S108). Here, when the index value Vi is less than or equal to the threshold value (YES in step S108), since the variation in the blastomere volume is relatively small, the state of the fertilized egg is determined to be good (step S109). On the other hand, when the index value Vi exceeds the threshold value (NO in step S108), the variation in the blastomere volume is large, and the state of the fertilized egg is determined to be bad (step S110).

[0082] Thus, it is theoretically possible to automatically determine the quality of a fertilized egg from the obtained index value Vi. However, since there is not enough knowledge on how to set the threshold value for the index value, it is considered difficult at present to make a definite automatic determination based only on the calculation results. Therefore, even if a provisional determination result is shown as reference information, the final determination can be left to the user by presenting the index value Vi itself to the user.

[0083] In addition, in order to assist the user in comprehensively evaluating the fertilized egg from various viewpoints, it is also effective to present a combination of index values obtained by a plurality of calculation methods. For example, a presentation method in which some of the index values obtained by the calculation methods of (Equation 1) to (Equation 6) are listed together may be used, or a method of presenting the total value thereof as one index value may be used. In any of the calculation methods of the above equations, the better the uniformity of the blastomeres, the smaller the resulting numerical value. Therefore, even in the total value of several index values, the numerical value will be smaller if the uniformity of the blastomeres is good. On the other hand, when any of the index values indicates low uniformity, that is, a large value, the total value also becomes large. In other words, the total value becomes a small value only when each of the individual index values obtained by different methods shows a high degree of uniformity. Therefore, it becomes possible to comprehensively evaluate the fertilized egg using this total value.

[0084] In addition, it is also conceivable to perform a comprehensive evaluation by multivariate analysis using a plurality of index values. For this purpose as well, it is preferable to present various index values obtained by statistically processing the measured volumes of the individual blastomeres. As index values that can be used for such a purpose, in addition to those calculated by the above respective calculation methods, the following (Equation 7) to (Equation 10):

Number

[0085] Here, the operator Min(p1, …, pn) in (Equation 8) and (Equation 9) is assumed to represent a function that returns the minimum value among a plurality of numerical values p1, p2, …, pn. Among these, (Equation 7) represents the sum of the differences between the volume of each blastomere and the volume average value. Although the operation of dividing by the number of blastomeres X is omitted when compared with the index value obtained by (Equation 6), they are equivalent in terms of technical significance. Since normalization by the number of blastomeres is performed in (Equation 6), there is a difference in that the index value can be evaluated on the same scale regardless of the difference in the number of blastomeres such as the 4-cell stage and the 8-cell stage, but these can be used appropriately according to the purpose.

[0086] Further, (Equation 8) returns the minimum value among the differences between the volume of each blastomere and the volume average value as a result, and (Equation 9) returns the range taken by the difference between the volume of each blastomere and the volume average value. Also, (Equation 10) returns the standard deviation indicating the distribution of the differences between the volume of each blastomere and the volume average value as a result. By appropriately combining the index values obtained by these various calculation methods, it becomes possible to quantitatively and objectively represent the uniformity of the blastomeres.

[0087] As described above, in the image processing method of this embodiment, the blastomere regions occupied by individual blastomeres are specified from the three-dimensional image of the fertilized egg in the cleavage stage obtained by OCT imaging, and their numbers and volumes are measured. Then, an index value that quantitatively indicates the variation in volume between the blastomeres is calculated by a predetermined calculation formula. This index value can effectively support the user's evaluation work of the fertilized egg, and can also be important quantitative information for automating the evaluation work.

[0088] Also, when the number of extracted blastomeres does not conform to the regularity represented by a power of 2, some of those blastomeres with large volumes are bisected to adjust the number of blastomeres. Thereby, it is possible to eliminate the influence of the temporary size variation between blastomeres caused by the variation in the timing of cell division and provide quantitative information useful for the evaluation of the fertilized egg.

[0089] According to this embodiment, for example, when a user performs an evaluation operation of a fertilized egg based on the Veeck classification, by quantitatively showing the variation in the size of blastomeres, which is one of the evaluation indices, the operation can be effectively supported. For example, when there are a plurality of fertilized eggs, by quantitatively showing the degree of variation in blastomeres among them with a unified scale, the comparison of the states of fertilized eggs can be facilitated.

[0090] Note that the present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the gist thereof. For example, the image processing apparatus 1 of the above embodiment has a function of performing OCT imaging on the sample S and a function of executing image processing based on the imaging data. However, the image processing method of the present invention can also be executed by a computer device that does not have an imaging function itself and acquires imaging data obtained by imaging with another device having an imaging function. In order to enable this, the present invention may be implemented as a software program for causing a computer device to execute each processing step in FIG. 3.

[0091] Distribution of such a program can be performed, for example, in a form of downloading via a telecommunication line such as the Internet, and can also be performed by distributing a computer-readable recording medium in which the program is non-temporarily recorded. Further, by causing an existing OCT imaging device to read this program via an interface, it is also possible to implement the present invention by the device.

[0092] Also, in the above embodiment, the technique described in Patent Document 1 is used when extracting the blastomere region from the three-dimensional image of the fertilized egg. However, as long as it is a method capable of appropriately separating individual blastomeres constituting the fertilized egg, it is not limited to the above, and other methods may be used.

[0093] As described above by exemplifying specific embodiments, in the image processing method according to the present invention, the number X of the extracted blastomere regions satisfies the following formula: 2 N <X < 2 N+1 (N is a natural number) When represented by, among the cleavage regions, the (2 N+1 - X) ones with larger volumes may each be regarded as two cleavage regions with their volumes bisected, and the index value may be obtained. In the process of cleavage, it is common for the number of cells (blastomeres) to double successively. However, there is some variation in the timing of cell division. For this reason, depending on the timing of imaging, pre - division blastomeres and post - division blastomeres may be mixed in a still image, and the number of blastomeres may not be a power of 2. In such a case, by bisecting the blastomere with a larger volume on the premise that it is pre - division, the size of the post - division blastomere can be pseudo - represented. By doing so, it is possible to avoid the situation where the variation in size caused by the mixture of pre - and post - division blastomeres becomes a cause of mis - evaluation.

[0094] Also, for example, an index value based on the maximum value of the difference between the volume of each cleavage region and its average value or median value, and an index value based on the average value or median value of the volume may be obtained respectively. Regarding the size and its variation of each blastomere, they vary for each individual fertilized egg, and there may be cases where the state of the fertilized egg cannot be accurately represented by only a single index value. By combining multiple types of index values, it is possible to handle such situations. For example, a combination of an index value based on the maximum value of the difference and an index value based on the average value or median value is suitable for such purposes.

[0095] Also, for example, one index value may be obtained based on the maximum value of the difference and the average value or median value. For example, the sum of the index values obtained based on these two types of values can be used as the final index value. According to such a configuration, each value obtained by a plurality of calculation methods as a value quantitatively representing the state of the fertilized egg can be finally aggregated into a single index value, and a numerical value comprehensively representing the state of the fertilized egg can be obtained.

[0096] For example, the volume of each blastomere region may be normalized by the average value or median value of the volumes of all blastomere regions, and an index value may be obtained based on the normalized volume. According to such a configuration, it is possible to obtain an index value that changes as cleavage progresses or is not affected by differences in the sizes of individual blastomeres due to individual differences in fertilized eggs.

[0097] For example, the Watershed algorithm can be preferably applied to extract the blastomere region. Inside the fertilized egg, each blastomere is in a state of being in close contact with each other, and it is not easy to identify their boundaries in the OCT image. As a region segmentation process for a fertilized egg in the cleavage stage in which a plurality of cells are densely packed while maintaining their individual shapes to some extent, the Watershed algorithm that divides into a plurality of regions using the unevenness of the object surface as a clue is very useful.

[0098] Furthermore, in the image processing method and the fertilized egg evaluation method according to the present invention, a step of evaluating the state of the fertilized egg by comparing the obtained index value with a predetermined threshold value may be further provided. Although evaluating the variation in the size between blastomeres as an index when evaluating the state of a fertilized egg has been performed before, it has mainly been left to the subjective judgment of an observer. If this can be automatically performed based on quantitative information, the work burden on the observer can be greatly reduced, and it is possible to effectively suppress variations in the evaluation results.

Industrial Applicability

[0099] This invention is suitable for assisting the observation and evaluation work of a fertilized egg (embryo) in the cleavage stage. For example, it can be used for the purpose of assisting the work of evaluating the state of a cultured embryo and selecting a good fertilized egg with a higher pregnancy success rate in reproductive assisted medicine.

Explanation of Signs

[0100] 1 Image processing apparatus 10 Holding part 11 Sample container 20 Imaging unit 30 Control Unit 31 CPU 33 Signal Processing Unit 352 Display Unit B1, B2 Scoring Areas C Cell (Scored Ball) E Fertilized Egg (Embryo) S Sample

Claims

1. A step of obtaining three-dimensional image data representing a three-dimensional image of the fertilized egg, obtained by performing optical coherence tomography on the fertilized egg in the cleavage stage; A step of extracting, for each blastomere, a blastomere region occupied by the individual blastomeres in the three-dimensional image based on the three-dimensional image data; A step of obtaining the volume of each of the blastomere regions based on the three-dimensional image data; A step of obtaining an index value indicating the volume variation of the blastomeres based on the derivation result of the volume; comprising, wherein the index value is obtained based on the maximum value among the differences obtained between the volume of each of the individual blastomere regions and their average value or median value, an image processing method.

2. The number X of the extracted blastomere regions satisfies the following formula: 2 Ｎ < X < 2 Ｎ＋１ (N is a natural number) When represented by, among the divided sphere regions, the (2 Ｎ＋１ -X) ones with the largest volume are each regarded as two divided sphere regions obtained by bisecting the volume, and the index value is obtained. The image processing method according to claim 1.

3. The image processing method according to claim 1 or 2, wherein the index value based on the maximum value and the index value based on the average value or median value of the volumes of the individual blastomere regions are obtained.

4. The image processing method according to any one of claims 1 to 3, wherein the index value is obtained based on the maximum value and the average value or median value of the volumes of the individual blastomere regions.

5. The image processing method according to any one of claims 1 to 4, wherein the volume of each of the individual blastomere regions is normalized by the average value or median value of the volumes of all the blastomere regions, and the index value is obtained based on the normalized volume.

6. The image processing method according to any one of claims 1 to 5, further comprising a step of evaluating the state of the fertilized egg based on a comparison between the obtained index value and a predetermined threshold value.

7. The image processing method according to any one of claims 1 to 6, wherein the Watershed algorithm is used for extracting the blastomere regions.

8. A computer program for causing a computer to execute each step of the image processing method according to any one of claims 1 to 7.

9. A computer-readable recording medium on which the computer program according to claim 8 is non-temporarily recorded.

10. A step of performing optical coherence tomography on the fertilized egg in the cleavage stage to obtain three-dimensional image data representing a three-dimensional image of the fertilized egg; A step of extracting, for each blastomere, a blastomere region occupied by the individual blastomeres in the three-dimensional image based on the three-dimensional image data; A step of obtaining the volume of each of the blastomere regions based on the three-dimensional image data; A step of obtaining an index value indicating the volume variation of the blastomeres based on the derivation result of the volume; comprising, A method for evaluating a fertilized egg, which is determined based on the maximum value among the differences obtained between the volume of each of the above-mentioned cleavage sphere regions and their average value or median value.

11. The method for evaluating a fertilized egg according to claim 10, further comprising a step of comparing the obtained index value with a predetermined threshold value to evaluate the state of the fertilized egg.

Images