IMAGE PROCESSING METHOD, PROGRAM, RECORDING MEDIUM, AND IMAGE PROCESSING APPARATUS

Abstract

To derive automatically the number of cells forming trophectoderm from image data obtained by the OCT imaging of a blastocyst-stage embryo.SOLUTION: An image processing method according to the present invention includes a step of performing optical coherence tomography imaging of a blastocyst-stage embryo to acquire three-dimensional image data representing a three-dimensional image of the embryo, a step of extracting a region corresponding to a trophectoderm and an inner cell mass in the three-dimensional image based on the three-dimensional image data, a step of acquiring a thickness distribution representing the thickness of the trophectoderm for each position, a step of detecting and counting the number of projection sites projecting toward a blastocoel of the embryo from a surrounding area based on an acquisition result of the thickness distribution, and a step of calculating the number of cells constituting the trophectoderm based on a result of the counting, a total inner surface area of a zona pellucida of the embryo, and an area in the inner surface of a covered region covered with the inner cell mass.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to image processing for imaging an embryo (fertilized egg), particularly a blastocyst stage embryo having a cavity therein called a blastocoel, and analyzing its structure. [Background technology]

[0002] For example, in assisted reproductive technology (ART) for the purpose of infertility treatment, an embryo (fertilized egg) is fertilized outside the body and cultured for a certain period of time, and then returned to the body. However, the pregnancy success rate (in ART) is not necessarily high, and the mental and financial burden on the patient is also large. To solve this problem, methods to accurately judge the condition of the cultured embryos are being sought.

[0003] For example, in embryos that have developed to the blastocyst stage, a hollow structure called the blastocoel is formed inside the embryo, which is covered by the zona pellucida, and a layer of cells called the trophectoderm surrounds it. In particularly developed blastocysts, the inner cell mass grows to cover part of the trophectoderm. The number of cells that make up the trophectoderm is an indicator of the state of the embryo, and therefore a technique for counting this number non-invasively is required.

[0004] As technologies that are expected to meet such demands, the applicant of the present application has previously disclosed Patent Documents 1 and 2. Patent Document 1 describes a method for identifying and dividing the trophectoderm and inner cell mass from a three-dimensional image of an embryo (fertilized egg) captured by a non-invasive tomographic imaging technology such as optical coherence tomography (OCT). Patent Document 2 describes a technology that supports an observer in evaluating an embryo by representing an embryo captured by OCT in polar coordinates with its center of gravity as the origin and visualizing it in a two-dimensional map in which the reflected light intensity in each radial direction is converted into a luminance value. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2019-133429 A [Patent Document 2] JP 2019-132710 A Summary of the Invention [Problem to be solved by the invention]

[0006] The above-mentioned conventional techniques present the three-dimensional structure of the object of observation in an easily understandable manner to the observer, but do not yet allow automatic extraction of quantitative information that is effective in supporting observation and evaluation work. For example, no method has been disclosed for counting the number of cells that form the above-mentioned trophectoderm. In particular, for the trophectoderm in the part covered by the inner cell mass, it is not possible to identify them at a level where the shape of each cell can be distinguished, so counting is practically impossible at present. For these reasons, a technique is desired that can automatically determine the number of trophectoderm cells, including the part covered by the inner cell mass, from image data obtained by capturing an image of a blastocyst stage embryo.

[0007] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a technology that can automatically derive the number of cells that form the trophectoderm from image data obtained by OCT imaging of a blastocyst stage embryo. [Means for solving the problem]

[0008] In order to achieve the above object, one aspect of the present invention includes the steps of: performing optical coherence tomography imaging of a blastocyst stage embryo to obtain three-dimensional image data representing a three-dimensional image of the embryo; extracting an area corresponding to the trophectoderm and inner cell mass from the three-dimensional image based on the three-dimensional image data; obtaining a thickness distribution representing the thickness of the trophectoderm for each position; detecting and counting protruding areas that protrude toward the blastocoel of the embryo more than the surrounding areas based on the thickness distribution acquisition result; and calculating the number of cells that constitute the trophectoderm based on the counting result, the total area of ​​the inner surface of the zona pellucida of the embryo, and the area of ​​a covered area of ​​the inner surface that is covered by the inner cell mass. The number of cells is calculated by multiplying the number of protrusion sites detected per unit area of ​​the inner surface in the region not covered by the inner cell mass by the total area of ​​the inner surface. This is an image processing method.

[0009] Another aspect of the present invention is a program for causing a computer to execute each of the above steps. Another aspect of the present invention is a computer-readable recording medium non-temporarily storing the program.

[0010] In embryos that have developed to the point where an inner cell mass has been formed, the trophectoderm forms a layer of roughly one layer of cells that is attached to the inner surface of the zona pellucida that covers the surface of the embryo. Due to the internal structure of the cells, such as the cell nucleus, each cell has a structure in which the center is bulged compared to the periphery. For this reason, in a cell layer formed by cells connecting with each other laterally, unevenness occurs due to the shape of each cell. In particular, in a cell layer composed of one layer of cells that covers the inner blastocoel, such as an embryo (fertilized egg) at the blastocyst stage, protrusions corresponding to each individual cell appear in the layer. Therefore, the number of protrusions can be an indicator of the position and number of cells that make up the layer. In addition, the spacing between the protrusions can be an indicator of the size of each individual cell.

[0011] In the above invention, regions corresponding to the trophectoderm and inner cell mass are extracted based on three-dimensional image data of an embryo captured by OCT, and the above-mentioned protrusion sites are detected from the thickness distribution obtained for the trophectoderm. The thickness of the trophectoderm is not uniform microscopically, and in particular, the trophectoderm, which is composed of a single layer of cells and exposed toward the blastocoel, is thicker at the center of each cell and thinner at the periphery. Therefore, it can be said that there is a high probability that the detected protrusion site indicates the position of a single cell. For this reason, it is possible to determine the number of cells by counting the number of protrusion sites.

[0012] On the other hand, it is known that a cell layer similar to that of the trophectoderm in other parts is formed in the part covered by the inner cell mass. Although it is difficult to directly count this, the inventors of the present application have found that the density of the cells forming the cell layer is comparable to that of the trophectoderm in other parts.

[0013] Therefore, if the area ratio of the part of the trophectoderm exposed to the blastocoel to the part covered by the inner cell mass is known, it is possible to estimate the number of cells in the latter from the results of counting the number of cells in the former. In other words, it is possible to calculate the number of cells in the entire trophectoderm from the results of counting the protrusion sites, the total area of ​​the inner surface of the zona pellucida, and the area of ​​that region covered by the inner cell mass.

[0014] Another aspect of the present invention includes an image acquisition unit that acquires three-dimensional image data representing a three-dimensional image of a blastocyst stage embryo obtained by optical coherence tomography imaging of the embryo and two-dimensional image data representing a two-dimensional image of the embryo obtained by optical microscope imaging of the embryo, and an image processing unit that extracts an area corresponding to the trophectoderm and inner cell mass from the three-dimensional image based on the three-dimensional image data and the two-dimensional image data, acquires a thickness distribution representing the thickness of the trophectoderm for each position, detects protruding areas protruding toward the blastocoel of the embryo more than surrounding areas based on the acquisition result of the thickness distribution and counts the number of protruding areas, and calculates the number of cells constituting the trophectoderm based on the counting result, the total area of ​​the inner surface of the zona pellucida of the embryo, and the area of ​​a covered area of ​​the inner surface covered by the inner cell mass. The number of cells is calculated by multiplying the number of protrusion sites detected per unit area of ​​the inner surface in the region not covered by the inner cell mass by the total area of ​​the inner surface. The image processing device

[0015] In the invention configured in this way, it is possible to obtain three-dimensional image data by optical coherence tomography imaging and two-dimensional image data by optical microscope imaging for the same embryo. As will be described in detail later, by using three-dimensional image data and two-dimensional image data for the same embryo in combination, it is possible to divide the area corresponding to the trophectoderm and inner cell mass and the area corresponding to the zona pellucida in the three-dimensional image. By performing image processing similar to the above method on the area corresponding to the trophectoderm and inner cell mass extracted in this way, it is possible to calculate the number of cells in the entire trophectoderm. Effect of the Invention

[0016] As described above, according to the present invention, it is possible to automatically derive the number of cells that form the trophectoderm from image data obtained by imaging an embryo at the blastocyst stage. [Brief description of the drawings]

[0017] [Figure 1] 1 is a diagram illustrating an example of the configuration of an image processing device according to the present invention. [Diagram 2] FIG. 2 is a diagram showing a schematic structure of an embryo serving as a sample in this embodiment. [Diagram 3]4 is a flowchart showing image processing in the present embodiment. [Figure 4] FIG. 13 is a diagram showing the relationship between the focal depth and the area of ​​the extracted transparent zone. [Diagram 5] FIG. 2 is a diagram showing the correspondence relationship between an XYZ orthogonal coordinate system and a polar coordinate system. [Figure 6] FIG. 13 is a diagram illustrating a method for determining the thickness of the trophectoderm. [Figure 7] 13 is a flowchart showing a process for determining the thickness of the trophectoderm. [Figure 8] FIG. 1 illustrates an example of a two-dimensional mapping technique. [Figure 9] FIG. 13 is a diagram illustrating another example of a two-dimensional mapping method. [Figure 10] FIG. 1 illustrates an example of a three-dimensional mapping technique. [Figure 11] FIG. 2 is a diagram illustrating the principle of peak detection processing. [Figure 12] 13 is a flowchart showing a peak detection process. [Figure 13] FIG. 13 is a diagram showing an example of peak positions detected on a two-dimensional map. [Figure 14] FIG. 13 is a diagram showing an example of a peak position detected on a three-dimensional map. [Figure 15] 13 is a flowchart showing a process for calculating the number of cells. [Figure 16] FIG. 1 is a diagram illustrating the principle of a method for calculating the area of ​​a covered region. [Figure 17] 13 is a flowchart showing a modified example of image processing. [Figure 18] 1 is a flow chart showing a process for separating the inner cell mass and the trophectoderm. [Figure 19] FIG. 11 is a diagram showing a schematic diagram of changes to an object in this process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Fig. 1 is a diagram showing an example of the configuration of an image processing device suitable for executing the image processing method according to the present invention. This image processing device 1 captures tomographic images of a sample held in a liquid, for example an embryo (fertilized egg) cultured in a culture solution, and processes the resulting tomographic images to create a cross-sectional image showing the structure of one cross section of the sample. It also creates a three-dimensional image of the sample from a plurality of cross-sectional images. In order to unify the directions in the following figures, XYZ orthogonal coordinate axes are set as shown in Fig. 1. Here, the XY plane represents the horizontal plane. The Z axis represents the vertical axis, and more specifically, the (-Z) direction represents the vertical downward direction.

[0019] The image processing device 1 includes a holding unit 10. The holding unit 10 holds a sample container 11, which contains a sample S to be imaged, in a horizontal position. The sample container 11 is, for example, a flat-bottomed shallow dish-shaped container called a dish, in which a depression capable of holding a liquid is formed on the upper surface of a plate-like member. A medium M such as a culture fluid is poured into the sample container 11, and a fertilized egg as the sample S is held inside the medium M. In this example, a plurality of samples S are held in the dish as the sample container 11, but the present invention is not limited to this.

[0020] In this example, a plurality of samples S are held in a sample container 11 having a single recess, but the present invention is not limited to this. For example, the sample container 11 may be a well plate in which a plurality of recesses called wells are arranged in a single plate-like member. In this case, a plurality of samples S can be held in each of the plurality of wells. Also, for example, a plurality of dishes each holding a sample S may be held in the holder 10 in a horizontally aligned state and used for imaging.

[0021] The imaging section 20 is disposed below the sample container 11 held by the holding section 10. An optical coherence tomography (OCT) device capable of capturing a tomographic image of an object to be imaged in a non-contact and non-destructive (non-invasive) manner is used for the imaging section 20. As will be described in detail later, the imaging section 20, which is an OCT device, includes a light source 21 that generates illumination light for the object to be imaged, an optical fiber coupler 22, an object optical system 23, a reference optical system 24, a spectrometer 25, and a photodetector 26.

[0022] The imaging section 20 further includes a microscope imaging unit 28 for performing optical microscope imaging. More specifically, the microscope imaging unit 28 includes an imaging optical system 281 and an imaging element 282. The imaging optical system 281 includes an objective lens whose focus is adjusted to the sample S in the sample container 11. As the imaging element 282, for example, a CCD imaging element, a CMOS sensor, or the like can be used. As the microscope imaging unit 28, one capable of bright field imaging or phase contrast imaging is suitable. The object optical system 23 and the microscope imaging unit 28 are supported by a support member (not shown) that is horizontally movable, and the position in the horizontal direction can be changed.

[0023] The image processing device 1 further includes a control unit 30 that controls the operation of the device, and a drive section 40 that drives a movable section of the imaging unit 20. The control unit 30 includes a CPU (Central Processing Unit) 31, an A / D converter 32, a signal processing section 33, an imaging control section 34, an interface (IF) section 35, an image memory 36, and a memory 37.

[0024] The CPU 31 controls the operation of the entire device by executing a predetermined control program, and the control program executed by the CPU 31 and data generated during processing are stored in a memory 37. The A / D converter 32 converts signals output from the photodetector 26 and the image sensor 282 of the imaging unit 20 according to the amount of received light into digital data. The signal processing unit 33 performs signal processing, which will be described later, based on the digital data output from the A / D converter 32 to create an image of the object to be imaged. The image data created in this way is appropriately stored and saved in an image memory 36.

[0025] The imaging control unit 34 controls each unit of the imaging section 20 to perform imaging processing. Specifically, the imaging control unit 34 selectively positions the object optical system 23 for OCT imaging and the microscope imaging unit 28 for optical microscope imaging at a predetermined imaging position where the imaging sample S is accommodated in the imaging field of view. When the object optical system 23 is positioned at the imaging position, the imaging control unit 34 causes the imaging section 20 to perform OCT imaging processing, which will be described later, to acquire three-dimensional image data representing the three-dimensional structure of the sample S. On the other hand, when the microscope imaging unit 28 is positioned at the imaging position, the imaging control unit 34 causes the microscope imaging unit 28 to acquire two-dimensional image data representing a planar image of the sample S formed on the light receiving surface of the imaging element 282 by the imaging optical system 281.

[0026] The interface unit 35 is responsible for communication between the image processing device 1 and the outside. Specifically, the interface unit 35 has a communication function for communicating with external devices and a user interface function for receiving operation input from a user and notifying the user of various information. For this purpose, the interface unit 35 is connected to an input device 351, such as a keyboard, mouse, or touch panel, capable of receiving operation input related to function selection and operating condition setting of the device, and a display unit 352, such as a liquid crystal display, for displaying various processing results such as a tomographic image created by the signal processing unit 33 and a stereoscopic image created by the 3D reconstruction unit 34.

[0027] In the imaging unit 20, a low-coherence light beam containing a wide band of wavelength components is emitted from a light source 21 having a light-emitting element such as a light-emitting diode or a superluminescent diode (SLD). For the purpose of imaging a sample such as a cell, it is preferable to use, for example, near-infrared light so that the incident light reaches the inside of the sample.

[0028] The light source 21 is connected to an optical fiber 221, which is one of the optical fibers constituting the optical fiber coupler 22. The low-coherence light emitted from the light source 21 is branched by the optical fiber coupler 22 into light for two optical fibers 222 and 224. The optical fiber 222 constitutes an object-system optical path. More specifically, the light emitted from the end of the optical fiber 222 enters the object optical system 23.

[0029] The object optical system 23 includes a collimator lens 231 and an objective lens 232. Light emitted from an end of the optical fiber 222 enters the objective lens 232 via the collimator lens 231. The objective lens 232 has a function of converging the light (observation light) from the light source 21 onto the sample, and a function of collecting the reflected light emitted from the sample and directing it toward the optical fiber coupler 22. Although a single objective lens 232 is shown in the figure, a combination of multiple optical elements may be used. The reflected light from the object to be imaged enters the optical fiber 222 as signal light via the objective lens 232 and the collimator lens 231. The optical axis of the objective lens 232 is perpendicular to the bottom surface of the sample container 11, and in this example, the optical axis direction coincides with the vertical axis direction.

[0030] The CPU 31 gives a control command to the imaging control unit 34, and in response to this, the imaging control unit 34 moves the imaging unit 20 in a predetermined direction. More specifically, the imaging control unit 34 moves the imaging unit 20 in the horizontal direction (XY direction) and the vertical direction (Z direction). The horizontal movement of the imaging unit 20 changes the imaging range in the horizontal direction. Furthermore, the vertical movement of the imaging unit 20 changes the focal position in the optical axis direction of the objective lens 232 with respect to the sample S, which is the object to be imaged.

[0031] A part of the light incident on the optical fiber coupler 22 from the light source 21 is incident on the reference optical system 24 via the optical fiber 224. The reference optical system 24 includes a collimator lens 241 and a reference mirror 243, which together with the optical fiber 224 form a reference system optical path. Specifically, the light emitted from the end of the optical fiber 224 is incident on the reference mirror 243 via the collimator lens 241. The light reflected by the reference mirror 243 is incident on the optical fiber 224 as reference light.

[0032] The reference mirror 243 is supported by an advancing / retreating member (not shown) that operates in response to a control command from the imaging control unit 34, and is movable forward and backward in the Y direction. When the reference mirror 243 moves in the Y direction, that is, in the direction toward and away from the collimator lens 241, the optical path length of the reference light reflected by the reference mirror 243 is adjusted.

[0033] The reflected light (signal light) reflected by the surface or internal reflection surface of the sample and the reference light reflected by the reference mirror 243 are mixed in the optical fiber coupler 22 and enter the photodetector 26 via the optical fiber 226. At this time, interference occurs between the signal light and the reference light due to the phase difference, and the spectrum of the interference light varies depending on the depth of the reflection surface. In other words, the spectrum of the interference light has information on the depth direction of the imaged object. Therefore, the reflected light intensity distribution in the depth direction of the imaged object can be obtained by detecting the amount of light by dispersing the interference light for each wavelength and performing a Fourier transform on the detected interference signal. An OCT imaging technology based on such a principle is called Fourier Domain OCT (FD-OCT).

[0034] In the imaging section 20 of this embodiment, a spectroscope 25 is provided on the optical path of the interference light from the optical fiber 226 to the photodetector 26. As the spectroscope 25, for example, one using a prism or one using a diffraction grating can be used. The interference light is split into wavelength components by the spectroscope 25 and received by the photodetector 26.

[0035] By performing a Fourier transform on the interference signal output from the photodetector 26 in response to the interference light detected by the photodetector 26, the reflected light intensity distribution in the depth direction at the incident position of the illumination light in the sample, i.e., in the Z direction, can be obtained. By scanning the light beam incident on the sample container 11 in the X direction, the reflected light intensity distribution in a plane parallel to the XZ plane can be obtained, and from the result, a tomographic image of the sample with the plane as a cross section can be created. Since the principle is well known, a detailed description will be omitted.

[0036] In addition, by changing the beam incidence position in the Y direction in multiple steps and capturing a tomographic image each time, it is possible to obtain a large number of tomographic images of the sample in cross sections parallel to the XZ plane. By reducing the scanning pitch in the Y direction, it is possible to obtain image data with a resolution sufficient to grasp the three-dimensional structure of the sample. From these tomographic image data, it is possible to create three-dimensional image data (so-called voxel data) that correspond to the three-dimensional image of the sample.

[0037] In this way, the image processing device 1 has a function of acquiring an image of the sample S held in the sample container 11 together with the culture medium M. As images, two-dimensional image data obtained by optical microscope imaging, tomographic image data obtained by OCT imaging, and three-dimensional image data based thereon can be acquired.

[0038] Hereinafter, an embodiment of an image processing method according to the present invention that can be executed using the image processing device 1 configured as described above will be described. The image processing in this embodiment involves imaging a blastocyst stage fertilized egg (hereinafter simply referred to as an "embryo") as a sample S, and individually extracting from the image areas corresponding to the zona pellucida, trophectoderm, and inner cell mass, which are the main structures constituting the embryo. Finally, the number of cells that form the trophectoderm is determined.

[0039] Based on the data thus obtained, it becomes possible to effectively support the user (specifically, a doctor or an embryologist) in the evaluation of the embryo. For example, in the case of embryo culture for the purpose of infertility treatment, the image processing method of this embodiment can be applied to obtain knowledge for determining whether the culture is progressing well or not.

[0040] FIG. 2 is a diagram showing a schematic structure of an embryo as a sample in this embodiment. As is already known, when an egg is fertilized, cleavage begins, and a blastocyst is formed through a state called a morula. FIG. 2(a) shows a schematic internal structure of an embryo at the blastocyst stage. At the blastocyst stage, an embryo E has a cavity called a blastocoel B inside. More specifically, cells that have undergone cleavage are arranged as a thin layer on the surface of the embryo to form a trophectoderm T, and the internal space surrounded by the trophectoderm T forms the blastocoel B. In addition, an inner cell mass I in which a large number of cells are densely packed is formed in a part of the internal space.

[0041] Furthermore, a zona pellucida ZP is formed to cover the outer surface of the trophectoderm T. The zona pellucida ZP is a membrane with a substantially uniform thickness, mainly composed of glycoproteins. On the other hand, the trophectoderm T, which is formed by gathering a large number of cells, has a variety of thicknesses depending on the position, and is distributed so as to be attached to the entire inner surface of the zona pellucida ZP. As shown in FIG. 2(b), the trophectoderm T is formed by lining up a large number of cells C along the inner surface of the zona pellucida ZP. As the shapes and sizes of the individual cells C differ, the trophectoderm T becomes a layer with different thicknesses depending on the position. Note that in FIG. 2, the variation in thickness of the trophectoderm T is exaggerated more than it actually is.

[0042] These structures that compose the embryo E, namely, the zona pellucida ZP, the trophectoderm T, and the inner cell mass I, are important as focus areas in embryo evaluation. Therefore, a technology that automatically extracts areas corresponding to these structures from captured images is of great significance in supporting the user's embryo evaluation work. However, in tomographic images or three-dimensional images obtained by OCT imaging, there is almost no difference in brightness information between these structures. Therefore, it is difficult to accurately separate these structures by simple separation based on differences in brightness.

[0043] Thus, a technique for automatically separating the zona pellucida ZP, the trophectoderm T, and the inner cell mass I from an OCT image has not been established so far. In view of such a problem, Patent Document 2 previously disclosed by the applicant of the present application is capable of separating the regions corresponding to the trophectoderm and the inner cell mass from an OCT image. However, even with this technique, it cannot be said that the zona pellucida and the trophectoderm are clearly separated. Specifically, there is a possibility that the region extracted as the trophectoderm by this technique includes the region corresponding to the zona pellucida.

[0044] The first objective of the image processing in this embodiment is to separate the zona pellucida ZP and the trophectoderm T by analyzing the OCT image using information obtained from the two-dimensional image captured by optical microscopy, and ultimately to individually identify the areas occupied by the zona pellucida ZP, the trophectoderm T, and the inner cell mass I in the three-dimensional image. The second objective is to determine the number of cells that constitute the trophectoderm T thus separated.

[0045] In bright-field images or phase-contrast images obtained by optical microscopy, it is relatively easy to distinguish the zona pellucida ZP from the trophectoderm T based on the difference in brightness and texture. In a well-cultured blastocyst-stage embryo, the zona pellucida ZP has a generally uniform thickness. In other words, it can be considered that the zona pellucida ZP occupies the region of the embryo from its surface to a certain depth. From these facts, it is possible to extract the region corresponding to the zona pellucida ZP separately from the trophectoderm T by specifying the thickness of the zona pellucida ZP from the two-dimensional image data obtained by optical microscopy and specifying the region from the surface of the embryo E to a depth equivalent to the thickness of the zona pellucida ZP from the three-dimensional image data obtained by OCT imaging. In addition, for the separation of the inner cell mass I, for example, the method described in Patent Document 2 can be applied. As a result, the first objective can be achieved.

[0046] The trophectoderm T thus separated from other structures has irregularities resulting from the shapes of the individual cells C, as shown in Figure 2(b). More specifically, the surface of the trophectoderm T facing the blastocoel B has protrusions resulting from the shapes of the individual cells C. Therefore, by detecting such protrusion sites and counting their number, it is possible to know the number of cells C that make up the trophectoderm T. In this way, the second objective is achieved.

[0047] 3 is a flowchart showing image processing in this embodiment. This processing is realized by CPU 31 executing a control program prepared in advance and causing each unit of the device to perform a predetermined operation. When a specimen container 11 containing an embryo to be evaluated is set in holder 10 (step S101), optical microscope imaging and OCT imaging are performed by imaging unit 20 with the embryo as the imaging subject.

[0048] In Fig. 3, steps S102 to S105 represent data processing using optical microscope imaging and the two-dimensional image data obtained thereby. Meanwhile, steps S111 and S112 represent image processing using OCT imaging and the three-dimensional image data obtained thereby. These two processes may be performed in chronological order, and in this case, it is optional which process is performed first. Moreover, these processes may be performed in parallel to shorten the processing time.

[0049] First, an overview of the processing in steps S102 to S105 will be described. In this processing, two-dimensional image data of the embryo is obtained by imaging the embryo with an optical microscope, and the thickness of the zona pellucida is calculated based on the image data. Specifically, the microscope imaging unit 28 is positioned at the imaging position, and the focal position is changed and set in multiple steps in the depth direction (Z direction), and imaging is performed each time. As a result, a plurality of image sets with different focal depths, so-called Z-stack images, are obtained (step S102).

[0050] From each of these images, an area corresponding to the zona pellucida ZP is extracted (step S103). Then, from among the multiple images, the one that is best focused on the zona pellucida ZP is selected (step S104), and the thickness of the zona pellucida ZP is calculated based on the selected image (step S105). Details of these processes, i.e., the process of extracting the area corresponding to the zona pellucida ZP (step S103), the process of selecting the most focused image (step S104), and the process of calculating the thickness of the zona pellucida ZP (step S105), will be described later. Through the processes up to this point, the thickness of the zona pellucida ZP in the embryo E to be evaluated is known.

[0051] Meanwhile, in step S111, the object optical system 23 is positioned at the imaging position, and tomographic images of the embryo E are taken while scanning the imaging position, thereby acquiring three-dimensional image data of the embryo E. The brightness of each part shown in the three-dimensional image data is binarized using a predetermined threshold value, and the three-dimensional image of the embryo E is separated into a high-brightness region occupied by some relatively high-density structure (hereinafter referred to as a "structural region") and a lower-density, lower-brightness region. For example, the culture medium M has a low brightness in the OCT image.

[0052] As described above, structures occupying the structural region may include the zona pellucida ZP, the trophectoderm T, and the inner cell mass I. In steps S113 and S114, these are separated from each other. Specifically, in step S113, the region corresponding to the zona pellucida ZP is separated from the other structural regions. Then, in step S114, the trophectoderm T and the inner cell mass I are separated.

[0053] The processing up to this point (steps S101 to S114) achieves the first objective of identifying the areas occupied by the zona pellucida ZP, the trophectoderm T, and the inner cell mass I from the three-dimensional image of the embryo E. The subsequent processing of steps S115 to S120 is intended to achieve the second objective of counting the number of cells that make up the separated trophectoderm T. This processing will be described in detail later.

[0054] In the following, each elemental technology for executing each step of the above process (steps S103 to S105, S111) will be described in order. Note that the process for acquiring the Z-stack image in step S102 is well known, and therefore the description will be omitted. In addition, the technology described in Patent Document 2 can be applied to the process in step S114, and therefore the description will be omitted.

[0055] In step S103, a region corresponding to the zona pellucida ZP is extracted from two-dimensional image data obtained by capturing an image of the embryo E using an optical microscope. This process can be performed using an appropriate image processing technique. For example, a pattern recognition technique can be applied to extract a region having a specific characteristic from within an image. Specifically, a classification model is constructed by supervised learning using a previously acquired image of the zona pellucida as a teacher image, and the classification model can be used to segment an optical microscope image of the embryo E to be evaluated, thereby extracting a region corresponding to the zona pellucida ZP from the image.

[0056] In this embodiment, a known semantic segmentation method is used as an example of region division processing. The semantic segmentation method is a technique for labeling each pixel in an image using a classification model previously constructed by a deep learning algorithm. In this embodiment, this method can be used as follows.

[0057] First, an optical microscope image of an embryo in which the zona pellucida is captured with good image quality is prepared, and each pixel in the area corresponding to the zona pellucida in the image is labeled to indicate that. Then, a classification model is constructed by performing deep learning using the original microscope image as input data and the labeled image as correct answer data. By providing an unknown image as input data to the classification model thus constructed, an output image can be obtained in which the area corresponding to the zona pellucida in the image is labeled to that effect. By extracting such an area from the output image, it is possible to extract the zona pellucida as a result.

[0058] An example of a specific method for constructing a classification model will be described. This process can be executed by various computer devices having a function for displaying images and a function for receiving operation input from a user. For example, it can be executed by an image processing device 1 or a general-purpose computer device such as a personal computer.

[0059] First, an optical microscope image of an embryo captured in advance with the zona pellucida focused (in focus) is displayed. In the image processing device 1, the image can be displayed on the display unit 352. An instruction input from a user to specify an area corresponding to the in-focus zona pellucida for the image thus displayed is accepted. In this case, it is preferable that the user is an expert with sufficient knowledge of embryo images. Furthermore, when the image processing device 1 is used, an instruction input can be accepted via the input device 351.

[0060] A label indicating that the area is a transparent zone is added to the area designated as such. The image labeled in this way is used as the correct answer data, and the original image is used as input data to perform deep learning, thereby constructing a classification model for extracting transparent zones from the image. Labels other than transparent zones may be used as necessary.

[0061] This classification model is constructed using an image in which the transparent zone is in focus as an input image. Therefore, when this is applied to an unknown test image and the semantic segmentation method is executed, a region having strong characteristics of a transparent zone is extracted from the test image. When the transparent zone is extracted in this way, it becomes possible to calculate its thickness. In order to calculate the thickness with high accuracy, it is desirable to focus on the transparent zone over as wide an area as possible in the image. In other words, it can be said that it is desirable to calculate the thickness of the transparent zone based on an image in which the area of ​​the area extracted by the semantic segmentation method is as wide as possible. The area can be expressed by the number of pixels.

[0062] On the other hand, since the embryo E to be evaluated has a three-dimensional structure, the focus state on the zona pellucida ZP may not be necessarily good in an image captured by appropriately determining the focal depth. Therefore, from the Z-stack images acquired by varying the focal position in the depth direction (Z direction), one image with the largest area of ​​the region extracted as corresponding to the zona pellucida ZP is selected, and this image is used to calculate the thickness of the zona pellucida ZP.

[0063] In step S104, an image that is most focused on the transparent zone ZP is selected from the Z-stack images. In the semantic segmentation method in this embodiment, the area of ​​the transparent zone in focus is extracted from the image, so it can be said that the image with the largest area of ​​this area is highly likely to be the image that is most focused on the transparent zone ZP. Images that meet such conditions can be selected.

[0064] FIG. 4 is a diagram showing the relationship between the focal depth and the area of ​​the extracted zona pellucida. The upper part of FIG. 4(a) shows a schematic diagram of the relationship between optical microscope images (Z-stack images) captured at various focal positions in the depth direction and the area (area corresponding to the zona pellucida ZP) extracted from those images by the semantic segmentation method. The graph at the bottom of FIG. 4(a) illustrates the relationship between the focal position at the time of capturing and the area (number of pixels) of the extracted area. As shown in these, the area of ​​the extracted area is maximum in image Ia, which is the best focused and has the clearest zona pellucida ZP among the Z-stack images. In other words, the focal position corresponding to the image with the largest extracted area among the Z-stack images can be regarded as the focal position. Therefore, this image Ia may be selected as the image on which the thickness of the zona pellucida ZP is calculated.

[0065] However, for example, due to vibration during imaging, image blur may occur, and as a result, the apparent area of ​​the transparent zone may appear larger than it actually is. Figure 4(b) illustrates such a situation. For example, assume that image Ib is blurred due to vibration. In this case, the area extracted as corresponding to the transparent zone appears to be larger, and this position may be erroneously determined as the in-focus position.

[0066] In this embodiment, this problem is solved by using the brightness difference between the pixels on either side of the edge of the extracted region. That is, in a well-focused image, the boundary between the region corresponding to the transparent zone and its surrounding region is clear, and therefore it is considered that there is a clear contrast between the brightness of these regions. On the other hand, in an unfocused image, the boundary between these regions is not clear, and therefore the contrast is not clear.

[0067] For this reason, it is believed that it is possible to reduce the above-mentioned erroneous judgment by introducing an evaluation value that reflects the magnitude of the luminance change at the edge portion of the region, i.e., sharpness, instead of simply evaluating the size of the area (number of pixels) of the extracted region. There are various methods for quantifying the amount of edge change, and any of them can be appropriately selected and applied.

[0068] In this embodiment, as an example, the evaluation value is the area of ​​the extracted region multiplied by a coefficient reflecting the magnitude of the luminance change at the edge portion. For example, the coefficient can be the square of the luminance difference between pixels on either side of the edge. More specifically, the coefficient is calculated by squaring the difference between the average luminance of all pixels adjacent to the edge in the extracted region and the average luminance of all pixels adjacent to the edge outside the edge.

[0069] In this way, as shown in the graph at the bottom of Fig. 4(b), it is possible to reduce the risk that an increase in the extraction area due to the influence of the shift in image Ib will cause an erroneous determination of the focus position. Note that, although the luminance difference is squared here to make the coefficient a positive value, the absolute value of the luminance difference may be used as the coefficient instead.

[0070] The process for selecting an image focused on the transparent zone can be configured as follows. This process corresponds to step S104 in FIG. 3. For each image constituting the Z-stack image, a region corresponding to the transparent zone is extracted in step S103. To calculate the area of ​​the region thus extracted, the number of pixels belonging to the region is counted for each image. Then, the average brightness of adjacent pixels inside and outside the edge of the extracted region is calculated, and the difference is calculated. Based on these values, an evaluation value indicating the degree of focus of each image is calculated. Specifically, the evaluation value is calculated by multiplying the number of pixels in the extracted region by a coefficient expressed as the square of the brightness difference inside and outside the edge. The image with the largest evaluation value thus obtained is selected as the image most focused on the transparent zone.

[0071] Next, the process of step S105, in which the thickness of the zona pellucida ZP is calculated from one optical microscope image selected from the Z-stack images, will be described. For the selected image, an area corresponding to the zona pellucida ZP is extracted in step S103. In a well-cultured embryo E, it is considered that a circular area having a roughly constant width is extracted as an area corresponding to the zona pellucida ZP. In an image focused on the zona pellucida ZP, the width of the ring, that is, the distance between the inner edge and the outer edge of the ring, corresponds to the thickness of the zona pellucida ZP.

[0072] There are various methods for calculating the width of the ring. A simple method is the Distance function provided in the OpenCV (Open Source Computer Vision) library. There is a method that uses the Transform function. A pixel on the inner edge of the ring is selected as the target pixel and the Distance By applying the Transform function, it is possible to determine the distance from the pixel in question to the closest pixel on the outer edge of the ring. This distance represents the width of the ring at that position, i.e., the thickness of the transparent zone ZP. Conversely, it is equivalent to determine the shortest distance from a pixel on the outer edge of the ring to the inner edge, taking the pixel of interest as the pixel of interest. The average value of the widths determined in this way at each position on the ring can be used as the average thickness of the transparent zone ZP. Hereinafter, this average thickness will be represented by the symbol Ta.

[0073] The detailed processing contents of steps S103 to S105 have been described above. Next, the process of separating the zona pellucida in step S113 will be described. Here, in the three-dimensional image of the embryo E obtained by OCT imaging, the range from the surface to a depth Ta is regarded as the zona pellucida ZP. Therefore, by extracting only this range from the three-dimensional image, it is possible to extract only the structures corresponding to the zona pellucida ZP. On the other hand, by erasing the structures in this range, it is possible to extract structures other than the zona pellucida ZP, i.e., the trophectoderm T and the inner cell mass I.

[0074] For a three-dimensional image from which the region corresponding to the zona pellucida ZP has been erased, as described in Patent Document 2, a region segmentation process using, for example, a Local Thickness calculation is performed on the remaining structures, thereby separating the region corresponding to the trophectoderm T from the region corresponding to the inner cell mass I. In this way, the regions occupied by the zona pellucida ZP, trophectoderm T, and inner cell mass I can be individually extracted from the three-dimensional image of embryo E.

[0075] Next, a process (steps S115 to S120) for achieving the second object of this embodiment, that is, for determining the number of cells constituting the trophectoderm T, will be described. The outline of this process is to calculate the thickness distribution at each position of the trophectoderm T, detect the peak position in the determined thickness distribution, and count the number of significant peaks that correspond to individual cells. Each step will be described below in order.

[0076] In step S115, the three-dimensional image data expressed in the XYZ Cartesian coordinate system is converted to a polar coordinate representation. Since the embryo E is roughly spherical and hollow inside, the trophectoderm T has a shape close to a spherical shell. In order to more clearly express such a structure, it is preferable to express each position in polar coordinates (spherical coordinates) with the center of the embryo E as the origin. Therefore, as shown in FIG. 5, a coordinate conversion is performed from the XYZ Cartesian coordinate system to an rθφ polar coordinate system with the radius r and two argument angles θ and φ as coordinate variables.

[0077] 5 is a diagram showing the correspondence between the XYZ Cartesian coordinate system and the polar coordinate system. As is well known, the coordinates (x, y, z) of a point P in the Cartesian coordinate system and the coordinates (r, θ, φ) of a point P in the spherical coordinate system can be expressed by the following formula if the origin O is common: x = r sinθ cosφ y=r sinθ sinφ z=r cosθ There is a relationship between

[0078] Specifically, the center position of the embryo E is identified from the three-dimensional image data, and this center position is set as the origin O of the spherical coordinate system. This origin O does not need to coincide with the origin of the XYZ Cartesian coordinate system in the original signal data. Then, a coordinate conversion from Cartesian coordinates to spherical coordinates is performed by an appropriate conversion process. By performing the coordinate conversion in this manner, each position in the three-dimensional space identified by the XYZ Cartesian coordinate system can be expressed in polar coordinates.

[0079] The "center of the embryo" can be found based on the three-dimensional image data, for example, as follows. If the surface of the three-dimensional image of the embryo E represented by the three-dimensional image data can be regarded as a sphere, the center of gravity of the sphere in the image can be regarded as the center of the embryo. In three-dimensional image processing, a method for finding the center of gravity of a solid object is known, and such a method can be applied. In addition, for example, a sphere or ellipsoid that approximately represents the surface of the embryo may be specified, and the center of the approximate surface may be regarded as the center of the embryo.

[0080] In step S116, the thickness distribution of the trophectoderm T is obtained from the three-dimensional image data thus converted into polar coordinates. Specifically, the thickness of the trophectoderm T in one radial direction in the polar coordinate space is calculated, and this is repeated for various radial directions, thereby obtaining the thickness distribution of the trophectoderm T.

[0081] Fig. 6 is a diagram illustrating a method for determining the thickness of the trophectoderm. In the example shown in Fig. 6(a), a radius r1 extending from the origin O is considered, and the distance T1 between the inner surface of the trophectoderm T in the direction of the radius r1, i.e., the surface Si facing the blastocoel B, and the outer surface, i.e., the surface So facing the zona pellucida ZP, is defined as the thickness of the trophectoderm T in the radial direction.

[0082] 6(b), the thickness of the trophectoderm T in the direction of the radius r1 may be calculated by focusing on a point P1 where the radius r1 intersects with the inner surface Si of the trophectoderm T and taking the shortest distance between the point P1 and the outer surface So of the trophectoderm T as the thickness of the trophectoderm T in the direction of the radius r1. Alternatively, the thickness of the trophectoderm T in the direction of the radius r1 may be calculated by focusing on a point P2 where the radius r1 intersects with the outer surface So of the trophectoderm T and taking the shortest distance between the point P2 and the inner surface Si as the thickness of the trophectoderm T in the direction of the radius r1. To calculate the shortest distance, the Distance Transform function in the OpenCV library can be used, as in the above-described process for calculating the thickness of the zona pellucida ZP.

[0083] FIG. 7 is a flow chart showing a process for determining the thickness of the trophectoderm. First, one radial direction is selected (step S201). Specifically, one radial direction is specified by provisionally setting the values ​​of the deflection angles θ and φ appropriately. Next, a pixel corresponding to the position of the inner surface Si of the trophectoderm T in the radial direction (the position of point P1 shown in FIG. 6(b)) or the position of the outer surface So (the position of point P2 shown in FIG. 6(b)) in the polar coordinate space is specified as a pixel of interest (step S202). For later processing, it is desirable to store the distance from the origin O to the pixel of interest in association with the radial direction specified by the pair of deflection angles θ and φ. This information is represented by a function R(θ,φ).

[0084] Next, the distance between the inner surface Si and the outer surface So of the trophectoderm T in the radial direction, i.e., the thickness of the trophectoderm T, is calculated (step S203). The thickness can be calculated by the method shown in either FIG. 6(a) or FIG. 6(b).

[0085] The thickness of the trophectoderm T thus obtained is associated with one radial direction, and can therefore be expressed as a function of the deflection angles θ and φ. Hereinafter, the thickness of the trophectoderm T in one radial direction specified by a set of deflection angles θ and φ will be represented as T(θ,φ). The obtained thickness T(θ,φ) is stored in the memory 37 (step S204). By repeating the above process for various radial directions (step S205), a thickness distribution representing the thickness of the trophectoderm T in each direction can be obtained.

[0086] In step S117, a map is created that represents the spatial distribution of the thickness of the trophectoderm T thus obtained. The thickness distribution map may be a two-dimensional map or a three-dimensional map. As the two-dimensional map, a method of storing all directions in one map and a method of dividing the polar coordinate space into several spaces and storing each in a map are considered. Specific examples of creating these mapping methods are described below.

[0087] FIG. 8 is a diagram showing an example of a two-dimensional mapping method. In order to map each direction expressed in polar coordinates onto a two-dimensional plane, for example, a pseudocylindrical projection can be used. That is, as shown in FIG. 8(a), a projection similar to that used to represent the topography of the surface of the Earth, which is an approximately spherical body, on a flat map can be used, with the deflection angles θ and φ corresponding to latitude and longitude, respectively. Several such projection methods are known, and an appropriate one can be selected depending on the purpose.

[0088] Here, the Sanson projection is used, which is a map projection that can maintain the actual area on the map by treating the declination angle θ as latitude and the declination angle φ as longitude. At each point on the map specified by the combination of the declination angles θ and φ, a pixel having a brightness value corresponding to the thickness T(θ,φ) of the trophectoderm T calculated in each radial direction is placed. By displaying the thickness in each direction on the map in this way, the thickness distribution can be visualized. Figure 8(b) shows a map in which the thickness of the trophectoderm T at each point is represented by brightness, using isoluminance lines instead of directly representing it by brightness. It can also be said that the thickness in each direction is simply represented by contour lines. In addition, the thickness of the trophectoderm T can also be represented by color or shade.

[0089] In addition, the hatched region in the upper right corner of Figure 8(b) represents the region where the inner cell mass I is present. Although trophectoderm T is also present between the inner cell mass I and the zona pellucida ZP, the shape of the surface in contact with the inner cell mass I does not necessarily represent the shape of each individual cell. For this reason, in this embodiment, the region where the inner cell mass I is present is excluded from the target for deriving the thickness distribution.

[0090] As is also a problem in cartography, when the surface of an approximately spherical object is represented by a two-dimensional map, it is not possible to accurately represent all information. For example, in the case of the Sanson projection, which is a type of equal-area pseudocylindrical projection, the area of ​​the earth's surface can be accurately represented, but distance and direction cannot always be represented correctly. In the case of mapping according to this embodiment, distortion becomes large, especially at the periphery of the map plane.

[0091] Fig. 9 is a diagram showing another example of a two-dimensional mapping method. In Fig. 9(a), a three-dimensional image of the trophectoderm T, which is represented as an approximately spherical surface in a polar coordinate space, is divided into four by two planes that pass through the origin O and are perpendicular to each other. Then, each of the divided sections Ta to Td is represented on one map. At this time, in the map corresponding to the section Ta, the center of the divided sphere, more specifically, the intersection Ca of the sphere and a straight line that passes through the origin O and intersects with each of the two planes at an angle of 45 degrees, is set to be approximately the center of the map. The same is true for the other sections Tb, Tc, and Td.

[0092] In this way, as shown in Fig. 9(b), the trophectoderm T divided into four parts in the polar coordinate space is represented using only the central part of the map plane. Therefore, although the map has four sides, each map can accurately represent the thickness distribution of the trophectoderm T as a whole without using the peripheral parts where distortion becomes large.

[0093] In reality, it is more preferable to make each section partially overlap when considering the division of the sphere corresponding to the trophectoderm T, and to show the thickness distribution not only within section Ta but also to a certain range outside it when mapping, as shown by the dotted line in Fig. 9(b). Although it is divided for the convenience of mapping, the actual trophectoderm T is continuous beyond each section, and by mapping to a certain range outside the sections, the continuity of the thickness distribution at the outer edge of each section can be expressed.

[0094] In this sense, instead of dividing the map, it is possible to create multiple two-dimensional maps with different center positions and use only the central parts of the maps. The number of divisions is not limited to four as mentioned above and can be arbitrary.

[0095] The above-mentioned two-dimensional mapping method is suitable for applications such as viewing the entire thickness distribution on a screen or printed paper, but the accuracy inevitably decreases at the periphery. For this reason, three-dimensional maps are superior in terms of accuracy. In three-dimensional maps, it is possible to ensure the same level of accuracy in all directions in the polar coordinate space.

[0096] When displaying a three-dimensional map on a screen, for example, a map projected in one line of sight direction is temporarily displayed, and the line of sight direction can be changed according to a user operation, thereby improving user convenience. For example, if an animation display is used in which the line of sight direction changes in real time in conjunction with a user operation, the user can perform the evaluation work with a sensation as if he or she were directly observing the embryo E in front of his or her eyes.

[0097] Methods for three-dimensional mapping of the trophectoderm T include a method of imparting information representing the thickness distribution in each direction to an approximate sphere (or ellipsoid) that represents the general shape of the trophectoderm T, and a method of imparting thickness distribution information to a curved surface that corresponds to the actual three-dimensional shape of the trophectoderm T. In the latter case, there is a choice of whether the shape of the curved surface corresponds to the inner surface Si or the outer surface So of the trophectoderm T.

[0098] The outer surface So of the trophectoderm T is in contact with the zona pellucida ZP, and therefore its surface shape is relatively smooth. In this sense, the curved surface corresponding to the outer surface So of the trophectoderm T is essentially not very different from a sphere. On the other hand, the inner surface Si of the trophectoderm T has irregularities corresponding to the shapes of the individual cells that make up the trophectoderm T. For this reason, mapping onto the curved surface corresponding to the inner surface Si of the trophectoderm T has the advantage that it is easier to imagine the actual irregular shape.

[0099] As a method for visualizing thickness on the map surface, as in the case of two-dimensional mapping, a method of converting thickness into brightness and displaying it, a method of expressing it using contour lines (isoluminance lines), color coding, shading, etc. are applicable. Below, a case where mapping is performed on a spherical surface and a case where mapping is performed on a curved surface corresponding to the inner surface Si of the trophectoderm T are exemplified. As the mapping method, contour lines are used.

[0100] FIG. 10 shows an example of a three-dimensional mapping technique. FIG. 10(a) shows an example of mapping onto a spherical surface, and FIG. 10(b) shows an example of mapping onto a curved surface corresponding to the inner surface Si of the trophectoderm T. Here, the three-dimensional map is shown projected onto a two-dimensional plane, but in reality, mapping is performed on a three-dimensional curved surface. When displaying and outputting to a screen, it is possible to display the image with various different viewing directions. Note that in these maps, the hatched area corresponds to the inner cell mass I.

[0101] For example, the three-dimensional map in Fig. 10(a) can be created by arranging pixels with brightness corresponding to the thickness calculation result in the direction of the radius passing through each position on a sphere centered on the origin O in the polar coordinate space. That is, a pixel with a brightness value corresponding to the thickness T(θ,φ) is arranged at the intersection of the radius represented by the two deviation angles (θ,φ) and the sphere.

[0102] A three-dimensional map solves the problem of distortion that occurs in two-dimensional maps. Apart from the distortion problem, two-dimensional maps have the problem that the length of the radius, that is, information on the distance from the origin O, is not reflected in the map. In particular, when the shape of the sample S deviates significantly from a sphere, the unevenness displayed on the map may not adequately reflect the actual shape. With a three-dimensional map, it is possible to reduce such deviations by approximating the surface to, for example, a spheroid that is closer to the shape of the sample.

[0103] In particular, as shown in FIG. 10(b), by performing mapping on a curved surface that reflects the actual shape, a map that is more realistic can be created. Information required for mapping is the position (length of the radius) of the inner surface Si (or the outer surface So) of the trophectoderm T in each radial direction and the thickness of the trophectoderm T in that direction. These pieces of information have already been obtained as R(θ,φ) and T(θ,φ), respectively, in the process of calculating the thickness of the trophectoderm T (step S116 in FIG. 3). Therefore, a three-dimensional map is created by arranging a pixel having a brightness value corresponding to the thickness T(θ,φ) at a position represented by coordinates (R(θ,φ),θ,φ) in the polar coordinate space.

[0104] When the three-dimensional map shown in FIG. 10(b) is projected and displayed on a two-dimensional display screen, it is necessary to display the uneven shape of the map curved surface and the thickness of the trophectoderm T at each position in a distinguishable manner. In particular, the shape of the curved surface other than the outer periphery of the projected two-dimensional figure may be difficult to read from the displayed image. To avoid this problem, it is necessary to express the shape of the curved surface and the thickness at each point in different ways. For example, the shape of the curved surface may be represented by contour lines, while the thickness at each point may be represented by shading or color coding.

[0105] Returning to FIG. 3, we will continue to explain the method for counting the number of cells constituting the trophectoderm T from the thickness distribution mapped in this manner. The unevenness of the surface of the trophectoderm T visualized as described above is due to the shape of each individual cell. For this reason, when the thickness is expressed by brightness, for example, the areas scattered on the map that are brighter than the surroundings are considered to correspond to the protrusion sites corresponding to each individual cell in the trophectoderm T. Therefore, by extracting such areas that are thicker than the surroundings, that is, areas of high brightness, it is possible to estimate the location of each individual cell and to count the number of cells constituting the trophectoderm T.

[0106] For this reason, it is highly likely that the peaks appearing in the thickness profile of the trophectoderm T indicate protrusion sites corresponding to individual cells constituting the trophectoderm T. Thus, in step S118, the protrusion sites are indirectly detected by finding significant peaks in the thickness profile.

[0107] Fig. 11 is a diagram showing the principle of the peak detection process. Fig. 12 is a flowchart showing the peak detection process. As shown by the solid line in Fig. 11(a), when the relationship between the position in a certain direction on the map and the brightness (i.e., thickness) in that direction (thickness profile) is plotted, there is a variation in brightness for each position, and peaks with high brightness that stand out compared to the surroundings appear. These are candidates for peaks corresponding to the positions of cells.

[0108] First, such regions with higher brightness than the surroundings are extracted. Specifically, maximum value filtering is performed on the thickness profile shown by the solid line (step S301). By appropriately setting the window size and performing maximum value filtering, the peak width of the profile expands as shown by the dotted line in FIG. 11(a). Maximum value filtering increases the brightness value at positions other than the peak, but there is no change in the brightness value at the peak position. Therefore, such positions with no change in brightness value (shown by the dashed line in the figure) are extracted as peak candidates (step S302).

[0109] FIG. 11(b) shows the extracted peak candidates. When the brightness of a peak is extremely high or extremely low, the peak may be image noise or a peak not corresponding to a cell, which is different from the detection target. Therefore, thresholds Lth1 and Lth2 are set for the brightness value, and peak candidates whose brightness value exceeds the threshold Lth1 that defines the upper limit and whose brightness value is below the threshold Lth2 that defines the lower limit are eliminated (step S303). In FIG. 12, this process is described as "upper and lower limit process." In FIG. 11(b), black circles indicate peak candidates that are eliminated by this process. White circles indicate peak candidates that are not eliminated and remain.

[0110] In the present embodiment, in order to identify individual cells constituting the trophectoderm T, the thresholds Lth1 and Lth2 can be set to correspond to upper and lower limit values ​​of a reasonable size for a cell constituting the trophectoderm T. This makes it possible to exclude extremely large or small peak candidates that cannot be regarded as a single cell, thereby reducing errors in the cell counting results. Although an upper and lower limit values ​​are set for the peak brightness value here, it is also possible to set only one of them. Furthermore, the upper and lower limit processing may be omitted if it does not become a major error factor.

[0111] On the other hand, there are also reasonable cell sizes in the direction along the surface of the trophectoderm T. In other words, if the peak candidates properly represent individual cells, the distance between those peaks should be within a predetermined range. Therefore, the distance between adjacent ones of the remaining peak candidates is calculated (step S304), and if there are any peak candidates that are extremely close, one of them is regarded as noise and eliminated (step S305). It is more practical to eliminate the smaller peak. FIG. 11(c) shows an example of the result of eliminating peak candidates whose distance to adjacent peaks is less than the preset minimum distance Dmin.

[0112] In this way, the peaks indicated by white circles in Figure 11(c) are finally left. It is believed that the positions of these peaks represent the positions of the protrusion sites corresponding to individual cells. In other words, the processing up to this point has detected the protrusion sites present in the trophectoderm T surrounding the blastocoel B.

[0113] In this case, the principle is explained using a one-dimensional thickness profile to make the concept easier to understand, but the actual thickness profile is two-dimensional in the case where a two-dimensional map is used, and three-dimensional in the case where a three-dimensional map is used. Therefore, in the case where a two-dimensional map is used, the maximum value processing in step S301 becomes two-dimensional filtering. A circular filter is used in this case to apply a constant window size in each direction. The distance between peaks is also the distance on the two-dimensional map plane.

[0114] In a two-dimensional map, the length of the radius is not reflected, and the distortion increases toward the periphery. Therefore, strictly speaking, it cannot be said that an ideal filtering process is performed with a circular filter window. However, for the purpose of detecting peaks corresponding to the positions of individual cells, it is possible to obtain sufficient accuracy for practical purposes. In particular, in the method of dividing the object into multiple parts and mapping it as shown in Figure 9, the peripheral parts of the map where distortion increases are not used, so it is possible to sufficiently suppress errors due to such factors. For the range indicated by the dotted line in Figure 9(b), by making it larger than at least the size of the filter window, it is possible to avoid errors caused by the filtering process.

[0115] Also, if the thickness profile is shown as a three-dimensional map, the filter window becomes a three-dimensional sphere, i.e. the maximum filtering becomes a spherical filtering, and the distance between peaks becomes the distance in three-dimensional space.

[0116] In a three-dimensional map (Figure 10(a)) in which the thickness of the trophectoderm T at each position is projected onto a spherical surface, information on the length of the radius from the origin O to the pixel of interest is not reflected in the processing. This has the effect of simplifying the processing, but it can also be a source of error in the filter processing. On the other hand, in a three-dimensional map (Figure 10(b)) projected onto a curved surface corresponding to the surface shape of the trophectoderm T, the distance from the origin O is reflected in the position of the pixel of interest, so it is possible to eliminate errors and perform appropriate peak detection by using a spherical filter processing centered on the pixel of interest.

[0117] FIG. 13 is a diagram showing an example of peak positions detected on a two-dimensional map. As shown by the dotted line in FIG. 13(a), among the regions where the thickness of the trophectoderm T locally protrudes compared to the surroundings, those that meet the above-mentioned conditions are detected as significant peaks (black circles), and these peaks are considered to correspond to each of the cells that constitute the trophectoderm T. FIG. 13(b) shows only the detected significant peaks plotted on a two-dimensional map. In this way, when significant peaks corresponding to individual cells are detected, the number of protruding sites is calculated by counting them (step S119), and the number of cells that constitute the trophectoderm T can be indirectly known.

[0118] Although it is difficult to know the distribution of the cells that make up the trophectoderm T in the region occupied by the inner cell mass I, it is possible to estimate the number of cells in the entire embryo E by assuming that the cells are distributed at the same density as in other regions. In other words, the cell density is calculated by dividing the number of counted cells by the surface area of ​​the trophectoderm T excluding the part in contact with the inner cell mass I, and the total number of cells that make up the entire trophectoderm T can be estimated by multiplying the surface area of ​​the trophectoderm T including the part in contact with the inner cell mass I by the cell density.

[0119] FIG. 14 is a diagram showing an example of peak positions detected on a three-dimensional map. As in the above, in a three-dimensional map, among the regions where the thickness of the trophectoderm T locally protrudes compared to the surroundings, those that meet the conditions are detected as significant peaks (black circles) corresponding to cells. By counting these over the entire map surface, it is possible to estimate the number of cells that make up the trophectoderm T. The region occupied by the inner cell mass I can be handled in the same manner as in the case of a two-dimensional map.

[0120] Through the processing up to this point, the number of cells was counted for the portion of the trophectoderm T formed along the inner surface of the zona pellucida ZP that is exposed facing the blastocoel B. On the other hand, the portion covered by the inner cell mass I was not included in the above counting results. It has been found that trophectoderm T similar to the exposed portion is also formed between the inner cell mass I and the zona pellucida ZP. However, no method has been established for image processing to separate the trophectoderm T and the inner cell mass I that can reproduce the shape of each individual cell. For this reason, the above-mentioned cell counting method cannot be applied to the trophectoderm T covered by the inner cell mass I.

[0121] According to the findings of the present inventors, the density of cells that form the trophectoderm T between the inner cell mass I and the zona pellucida ZP is not significantly different from the density in the part not covered by the inner cell mass I. From this, by assuming that cells are distributed under the inner cell mass I at the same cell density as in the exposed part, it is possible to estimate the number of cells in the part of the trophectoderm T covered by the inner cell mass I. Specifically, this is as follows.

[0122] The cells that make up the trophectoderm T form a thin layer that adheres to the inner surface of the zona pellucida ZP. In the following explanation, the area of ​​the inner surface of the zona pellucida ZP that is covered by the inner cell mass I will be referred to as the "covered area," and the other area, that is, the area that is covered by the trophectoderm T and that is exposed toward the blastocoel B, will be referred to as the "exposed area."

[0123] The number of cells of the trophectoderm T that is formed to cover the exposed area is determined by the counting method described above. By dividing this number by the area of ​​the exposed area, it is possible to calculate the density per unit area of ​​the cells that form the trophectoderm T. Then, by multiplying the cell density thus determined by the area of ​​the entire inner surface of the zona pellucida ZP, it is possible to calculate the total number of cells that form the trophectoderm T on the inner surface of the zona pellucida ZP.

[0124] Therefore, if the surface area of ​​the entire inner surface of the zona pellucida ZP and the surface area of ​​the exposed area of ​​the zona pellucida ZP are known, the total number of cells can be calculated using the above counting results. (Total number of cells) = (total surface area of ​​inner surface) x (counting result) / (surface area of ​​exposed area) This makes it possible to calculate the total number of cells that make up the trophectoderm T.

[0125] In addition, since the inner surface of the zona pellucida ZP is divided into an exposed area and a covered area, if any two of the areas of the entire inner surface, the exposed area, and the covered area are known, the other one can be automatically calculated. Also, as is clear from the above formula, it is sufficient to know the area ratio between the surface area of ​​the entire inner surface and the surface area of ​​the exposed area. Based on these ideas, the above principle formula can be appropriately modified, but they are technically equivalent and all are included in the technical idea of ​​this embodiment. For example, the number of cells can be calculated by the following process.

[0126] Fig. 15 is a flow chart showing the process of calculating the number of cells. This process corresponds to step S120 in Fig. 3. In this process, the area (total area) of the entire inner surface of the zona pellucida ZP and the area of ​​the covered region covered by the inner cell mass I are calculated (steps S401, S402). The cell density in the region is calculated by dividing the result of counting the number of cells in the region by the area of ​​the exposed region, which is represented by the difference between these (step S403). The number of cells in the entire trophectoderm T, including the exposed region and the covered region, can be calculated by multiplying this cell density by the total area (step S404).

[0127] Next, a method for calculating the surface area of ​​the zona pellucida ZP will be described. By the image processing shown in Figure 3, the three-dimensional image of the embryo is divided into an area corresponding to the zona pellucida ZP, an area corresponding to the trophectoderm T, and an area corresponding to the inner cell mass I. Therefore, the positions occupied by the pixels on the boundaries of these areas in the three-dimensional image space are identified.

[0128] From this, the number of pixels at the boundary between the region corresponding to the zona pellucida ZP and the region corresponding to the trophectoderm T and inner cell mass I has the meaning of information representing the total area of ​​the inner surface of the zona pellucida ZP. Since pixels (voxels) in a three-dimensional image have a certain size in each of the three-dimensional directions, the number of voxels has the dimension of volume. Therefore, if the number of voxels is divided by the length of one side of the voxel, it becomes the dimension of area, and the area to be calculated can be approximately expressed by this value. Note that since the voxel size is constant, the number of pixels (voxels) may be used as an index of area for the purpose of calculating the area ratio.

[0129] In this way, the total area of ​​the inner surface of the zona pellucida ZP can be calculated. On the other hand, the area of ​​the covered region can be calculated, for example, as follows.

[0130] FIG. 16 is a principle diagram showing a method for calculating the area of ​​the covered region. In a three-dimensional image of an embryo E, consider half-lines extending from the center O of the embryo in any radial direction. One of these lines passing through the region corresponding to the inner cell mass I is the half-line L1 shown in FIG. 16. In this way, a voxel corresponding to point Q, which is on the half-line extending from the germinal center O and passing through the region corresponding to the inner cell mass I, and is located at the intersection of the half-line and the inner surface of the zona pellucida ZP, can be said to be a voxel in the covered region. By counting the number of voxels that meet these conditions, the area (more strictly speaking, the volume) of the covered region can be calculated.

[0131] As a special case, for example, when the inner surface of the zona pellucida ZP can be considered as a spherical shell, the area of ​​the covered region can be simply expressed by the solid angle α (unit: steradian) when the inner cell mass I is projected from the center O of the embryo E. Specifically, for example, of the half lines that can be taken in various directions from the center O of the embryo, such as half lines L2 and L3 shown in Figure 16, it is possible to identify the half lines that pass through the periphery of the inner cell mass I, and to determine the solid angle α as the opening angle of the cone surface formed by the assembly of these lines.

[0132] On the other hand, since the total area of ​​the inner surface of the spherical shell, the zona pellucida ZP, can be expressed in 4π steradians, the area ratio can be expressed using the solid angle α and 4π. Note that in order to express it in the actual dimensions of area, it is necessary to take into account the distance from the center O to the inner surface of the zona pellucida ZP, but this is not necessary when discussing the area ratio.

[0133] In the above embodiment, the method described in Patent Document 2 is used to separate the trophectoderm T and the inner cell mass I from a three-dimensional image of the embryo E. In this method, the peak corresponding to the trophectoderm T and the peak corresponding to the inner cell mass I are separated from the interference signal acquired by optical coherence tomography, thereby dividing these two regions. However, it is not easy to separate the two peaks with high accuracy, especially when the inner cell mass I is thin, and this may result in a decrease in accuracy when identifying the periphery of the covered region. This also affects the calculation result of the area of ​​the covered region, resulting in a decrease in the accuracy of calculating the number of cells.

[0134] This problem can be avoided by modifying the process for separating the inner cell mass I and the trophectoderm T. The basic idea is that while the trophectoderm T is a thin layer along the inner surface of the zona pellucida ZP, the inner cell mass I is a thicker and larger mass, so that of the structures formed along the inner surface of the zona pellucida ZP, "the largest one having a certain thickness or more" is regarded as the inner cell mass I. Specifically, for example, this can be done as follows.

[0135] Fig. 17 is a flowchart showing a modified image processing. In this modified example, compared with the flowchart shown in Fig. 3, the process for separating the inner cell mass I and the trophectoderm T (step S114), which was executed after step S113 in Fig. 3, has been moved after step S117 and is described as step S121. Furthermore, the process content is also different from that described above.

[0136] Except for this point, the processing contents are the same as those shown in Fig. 3. Therefore, the same processing as in Fig. 3 will be assigned the same step numbers and explanations will be omitted. Note that there is no substantial difference in the processing contents for steps S116 and S117, but they differ from the above embodiment in that the inner cell mass I is handled in a state in which it is not distinguished from the trophectoderm T. The processing contents in step S121 are as follows.

[0137] Fig. 18 is a flow chart showing the process for separating the inner cell mass and the trophectoderm. Fig. 19 is a schematic diagram showing the changes in the object during this process. This process is performed on a three-dimensional image of an embryo E from which the region corresponding to the zona pellucida ZP has been separated, as shown in Fig. 19(a). The object contained in the three-dimensional image at this time is an object (I+T) in which the inner cell mass I and the trophectoderm T are integrated without being separated.

[0138] First, the average thickness of the region of the object (I+T) corresponding to the trophectoderm T is obtained (step S501). Since the trophectoderm T and the inner cell mass I have not yet been separated at this point, strictly speaking it is not possible to obtain the thickness of only the trophectoderm T, but such precision is not required here. For example, a region of the object (I+T) whose thickness is equal to or less than a predetermined value may be assumed to be the trophectoderm T. Alternatively, the inner cell mass I and the trophectoderm T may be provisionally separated using the method described in Patent Document 2.

[0139] Next, the area corresponding to the calculated average thickness is removed from the outer surface of object (I+T) (step S502). The dashed line shown in FIG. 19(a) represents a position that is the average thickness inward from the outer surface of object (I+T). When the area outside this line is removed, only the part of object (I+T) that extends inward from the dashed line remains, as shown in FIG. 19(b). Of the objects that remain discretely in this way, the one with the largest volume is extracted, and this is regarded as the inner cell mass I (step S503).

[0140] Once the region corresponding to the inner cell mass I has been identified in this manner, this region is separated from the original object (I+T) (step S504), and the remaining region corresponds to the trophectoderm T, as shown in Figure 19(c).

[0141] According to this separation method, even when the inner cell mass I is thin, it is possible to satisfactorily separate it from the trophectoderm T. In particular, the inner cell mass I is thin at its periphery, and if this part cannot be properly distinguished from the trophectoderm T, the calculation error in the area of ​​the covered region described above will be large. The method shown here can be suitably applied to such a purpose.

[0142] As described above, in the image processing device 1 of this embodiment, the imaging section 20 functions as the "image acquisition section" of the present invention. Also, the control unit 30, particularly the CPU 31 and the signal processing section 33, function as the "image processing section" of the present invention.

[0143] The present invention is not limited to the above-mentioned embodiment, and various modifications other than those described above can be made without departing from the spirit of the present invention. For example, the image processing device 1 of the above-mentioned embodiment has a function of imaging the sample S with an OCT and an optical microscope, and a function of executing image processing based on the imaging data. However, the image processing method of the present invention can also be executed by a computer device that does not have an imaging function itself and acquires imaging data obtained by imaging with another device that has an imaging function. To make this possible, the present invention may be implemented as a software program for causing a computer device to execute the processing steps of FIG. 3 other than step S101.

[0144] Such a program can be distributed, for example, by downloading it via a telecommunications line such as the Internet, or by distributing a computer-readable recording medium in which the program is recorded. In addition, by having an existing OCT imaging device load the program via an interface, the present invention can be implemented by the device.

[0145] In addition, in the above embodiment, various processes are performed after converting the three-dimensional image data into polar coordinate data, but it is possible to detect the protrusion site similar to the above even without such coordinate conversion. For example, when using the Distance Transform function for thickness detection, it is not necessary to set the center (center of gravity) of the embryo, so it is possible to directly obtain the thickness from the three-dimensional image data of the trophectoderm represented in the Cartesian coordinate system. And, when using the three-dimensional map shown in FIG. 10(b), as long as the position of the inner surface or the outer surface is specified, it is possible to create a map in the XYZ Cartesian coordinate space. Therefore, if these processes are combined, it can be said that conversion to a polar coordinate system is not essential.

[0146] Also, for example, in the above embodiment, the zona pellucida and the trophectoderm are separated from the three-dimensional image of embryo E, and then the thickness distribution of the trophectoderm is obtained. However, for example, if the thickness of the zona pellucida can be considered constant, even if the thickness distribution is obtained without separating the zona pellucida and the trophectoderm, the thickness change that appears therein will substantially reflect the thickness of the trophectoderm. Furthermore, the method of separating the zona pellucida is not limited to the method using an optical microscope image as described above, and may be any method.

[0147] As described above by way of example of specific embodiments, in the image processing method according to the present invention, the number of cells can be calculated by multiplying the number of protrusion sites detected per unit area of ​​the inner surface of the zona pellucida that is not covered by the inner cell mass by the total area of ​​the inner surface. The number of protrusion sites per unit area represents the cell density in the trophectoderm. Therefore, it is possible to calculate the total number of cells that form the trophectoderm by multiplying the cell density by the total area of ​​the inner surface.

[0148] Also, for example, after extracting the regions corresponding to the trophectoderm and the inner cell mass from the three-dimensional image all at once, the region can be divided into a region corresponding to the trophectoderm and a region corresponding to the inner cell mass. The inner cell mass is formed so as to cover a part of the trophectoderm formed on the entire inner surface of the zona pellucida. Therefore, the boundary between the zona pellucida and the trophectoderm is not necessarily clear. On the other hand, the zona pellucida formed so as to cover the outside of these is a membrane with a roughly uniform thickness. Therefore, it is possible to divide the regions more efficiently by first extracting the regions corresponding to the trophectoderm and the inner cell mass all at once to separate the region corresponding to the zona pellucida, and then dividing the trophectoderm and the inner cell mass.

[0149] Specifically, for example, a region having a thickness equivalent to the average thickness of the trophectoderm is removed from the outer surface of the collectively extracted regions, and the largest volume of the remaining regions is determined to be the region corresponding to the inner cell mass. When a certain thickness is removed from the outer surface of the region containing the trophectoderm and inner cell mass, the region is left with discrete portions protruding toward the blastocoel. Since the largest of these is highly likely to be the inner cell mass, the above method can be used to identify the region occupied by the inner cell mass. Using this principle, it is possible to accurately divide the trophectoderm and the inner cell mass.

[0150] For example, based on two-dimensional image data obtained by imaging an embryo with an optical microscope, it is possible to divide the area corresponding to the trophectoderm and inner cell mass in a three-dimensional image into an area corresponding to the zona pellucida and an area corresponding to the zona pellucida. In a three-dimensional image obtained by OCT imaging, there is no clear difference in brightness between the zona pellucida and the trophectoderm attached to its inner surface, and it is not easy to separate them. On the other hand, in an optical microscope image, it is easier to distinguish the zona pellucida from other areas due to the difference in brightness. By utilizing this, it is possible to accurately separate the zona pellucida from other areas in a three-dimensional image based on two-dimensional image data obtained by imaging the same embryo with an optical microscope.

[0151] For example, polar coordinate data is created from the three-dimensional image data, in which the position of each point of the embryo is expressed as a position in a polar coordinate space, and the thickness distribution of the trophectoderm can be obtained based on the polar coordinate data. In particular, it is preferable to set the center of gravity of the embryo as the origin of the polar coordinate space. With this configuration, data processing for blastocyst stage embryos having a shape close to a sphere or spheroid can be facilitated.

[0152] In this case, for example, a thickness profile may be created that associates the radial direction in the polar coordinate space with the thickness of the trophectoderm determined in the radial direction, and protrusion sites may be detected based on this. With this configuration, it is possible to easily detect protrusion sites whose thickness protrudes relative to the surroundings from the change in the thickness of the trophectoderm shown in the thickness profile.

[0153] Specifically, for example, a maximum value filter process is performed on the thickness profile, and the position where the value does not change before and after the process can be determined to be the position of the protrusion site. Since the thickness of the trophectoderm is greater than the surrounding area at the position of the protrusion site, the thickness value does not change even after the maximum value filter process. On the other hand, in an area where there are thicker parts around, the value after the maximum value filter process changes due to the influence of the thickness. By utilizing this, it is possible to detect a part that is thicker than the surrounding area.

[0154] For example, the thickness profile can be represented by a three-dimensional map in which, at positions corresponding to the inner or outer surface of the trophectoderm, pixels having brightness values ​​corresponding to the thickness of the trophectoderm at those positions are arranged. Also, for example, the thickness profile can be represented by a three-dimensional map in which, at each position on the surface of an approximate sphere obtained by approximating the trophectoderm as a sphere in polar coordinate space, pixels having brightness values ​​corresponding to the thickness of the trophectoderm in the radial direction passing through that position are arranged. In these cases, a maximum value filter process can be performed as a spatial filter process for a spherical region centered on the pixel of interest.

[0155] On the other hand, it is also possible to represent the thickness profile by a two-dimensional map in which pixels having brightness values ​​corresponding to the thickness of the trophectoderm in one radial direction specified in the polar coordinate space by the two deviation angles are arranged on a coordinate plane whose coordinate axes are the two deviation angles represented by the polar coordinate data. In this case, a maximum value filter process can be executed as a two-dimensional filter process for a circular region centered on a pixel of interest on the two-dimensional map. As a method for representing the thickness distribution in a three-dimensional space by a two-dimensional map, for example, a pseudocylindrical projection method can be used.

[0156] Also, for example, the polar coordinate space may be divided into a plurality of spaces by a plane passing through the origin, and the creation of the two-dimensional map and the detection of the protruding portion may be performed for each divided space individually. When expressing the shape of an object in a three-dimensional space by a two-dimensional map, it is inevitable that the object on the map will be distorted. By dividing the space and creating maps individually, it is possible to avoid large distortions in specific locations, and to improve the detection accuracy of the thickness distribution. [Industrial Applicability]

[0157] The present invention is suitable for assisting in the observation and evaluation of blastocyst stage embryos (fertilized eggs), and can be used, for example, to assist in the evaluation of the condition of cultured embryos, for the purpose of selecting better embryos with a higher chance of successful pregnancy in assisted reproductive technology. [Explanation of symbols]

[0158] 1 Image processing device 10 Holding part 11 Sample container 20 Imaging unit (image acquisition unit) 30 Control unit (image processing section) 31 CPU (image processing unit) 33 Signal processing section (image processing section) B blastocoel E-embryo I Inner cell mass S sample T Trophectoderm ZP zona pellucida

Claims

1. performing optical coherence tomography imaging of a blastocyst stage embryo to obtain three-dimensional image data representative of a three-dimensional image of the embryo; extracting an area corresponding to the trophectoderm and the inner cell mass from the three-dimensional image based on the three-dimensional image data; Obtaining a thickness distribution representing the thickness of the trophectoderm by position; Based on the thickness distribution, detecting and counting the number of protruding portions protruding toward the blastocoel of the embryo relative to the surrounding areas; Calculating the number of cells constituting the trophectoderm based on the results of the counting, the total area of ​​the inner surface of the zona pellucida of the embryo, and the area of ​​the covered region of the inner surface that is covered by the inner cell mass; Equipped with An image processing method in which the number of cells is calculated by multiplying the number of protrusion sites detected per unit area of ​​the inner surface in a region not covered by the inner cell mass by the total area of ​​the inner surface.

2. 2. The image processing method according to claim 1, further comprising the steps of: extracting from said three-dimensional image a region corresponding to said trophectoderm and said inner cell mass collectively; and then dividing said region into a region corresponding to said trophectoderm and a region corresponding to said inner cell mass.

3. 3. The image processing method according to claim 2, wherein an area having a thickness equivalent to an average thickness of the trophectoderm is removed from the outer surface of the collectively extracted areas, and the area having the largest volume among the remaining areas is determined to be the area corresponding to the inner cell mass.

4. 4. An image processing method according to claim 1, further comprising dividing the three-dimensional image into an area corresponding to the trophectoderm and inner cell mass and an area corresponding to the zona pellucida, based on two-dimensional image data obtained by imaging the embryo with an optical microscope.

5. creating polar coordinate data representing the positions of each point of the embryo as positions in a polar coordinate space from the three-dimensional image data; 5. The image processing method according to claim 1, further comprising the step of determining a thickness distribution of the trophectoderm based on the polar coordinate data.

6. The image processing method according to claim 5 , wherein the center of gravity of the embryo is set as the origin of the polar coordinate space.

7. The image processing method according to claim 5 or 6, further comprising detecting the protrusion site based on a thickness profile that associates a radial direction in the polar coordinate space with a thickness of the trophectoderm determined in the radial direction.

8. A step of extracting an area corresponding to the trophectoderm and the inner cell mass from a three-dimensional image of a blastocyst stage embryo based on three-dimensional image data representing the three-dimensional image of the embryo, the area corresponding to the trophectoderm and the inner cell mass from the three-dimensional image; Obtaining a thickness distribution representing the thickness of the trophectoderm by position; Based on the thickness distribution, detecting and counting the number of protruding portions protruding toward the blastocoel of the embryo relative to the surrounding areas; a step of calculating the number of cells constituting the trophectoderm by calculating the product of the number of protrusion sites detected per unit area of ​​the region not covered by the inner cell mass on the inner surface and the total area of ​​the inner surface based on the results of the counting, the total area of ​​the inner surface of the zona pellucida of the embryo, and the area of ​​the covered region of the inner surface that is covered by the inner cell mass; A program for causing a computer to execute the above.

9. A computer-readable recording medium on which the program according to claim 8 is non-transitoryly recorded.

10. an image acquisition unit that acquires three-dimensional image data representing a three-dimensional image of a blastocyst stage embryo obtained by optical coherence tomography imaging of the embryo and two-dimensional image data representing a two-dimensional image of the embryo obtained by optical microscopy imaging of the embryo; an image processing unit that extracts regions of the three-dimensional image corresponding to the trophectoderm and inner cell mass based on the three-dimensional image data and the two-dimensional image data, obtains a thickness distribution that indicates the thickness of the trophectoderm for each position, detects protruding regions that protrude toward the blastocoel of the embryo more than the surrounding regions based on the thickness distribution result, and counts the number of such protruding regions, and calculates the number of cells that constitute the trophectoderm based on the counting result, the total area of ​​the inner surface of the zona pellucida of the embryo, and the area of ​​the covered region of the inner surface that is covered by the inner cell mass; Equipped with An image processing device wherein the number of cells is calculated by multiplying the number of protrusion sites detected per unit area of ​​the region of the inner surface that is not covered by the inner cell mass by the total area of ​​the inner surface.

Images