Information processing device, cell evaluation system, information processing method, program, and learning method

Abstract

This information processing device has at least one processor. The processor acquires a phase image generated on the basis of a hologram of a cell captured using a digital holography technique, and derives evaluation information indicating the possibility of the cell changing to a predetermined form, on the basis of distribution information that indicates the distribution of phase values in the phase image.

Description

Information processing device, cell evaluation system, information processing method, program, and learning method

[0001] The disclosed technology relates to an information processing device, a cell evaluation system, an information processing method, a program, and a learning method.

[0002] The following techniques are known for cell evaluation methods using digital holography. For example, International Publication No. 2022 / 018730 describes performing holographic imaging of a heterogeneous cell population flow to enable label-free quantitative imaging of the cell flow, and automatically processing the image data from the holographic imaging to identify and classify specific types of cells during the cell flow. The step of performing holographic imaging of the cell flow includes performing digital holographic microscopy and quantitative phase microscopy to measure a quantitative phase profile of the cells that indicates the optical path delay profile of the cells. The step of automatically processing the image data includes performing classification based on 2D and 3D morphological quantitative features and optical topological features of the cells during the cell flow. Morphological quantitative features include area, diameter, eccentricity, and solidity. Optical topological features include mean, energy, volume, dry mass, variance, kurtosis, skewness, contrast, entropy, uniformity, and correlation.

[0003] U.S. Patent Application Publication No. 2019 / 226972 describes a method for automatically classifying micro-objects, including the steps of acquiring digital holographic data from a sample imaged with a common-path sharing digital holographic microscope, the digital holographic data including a hologram of at least one micro-object within the sample, and determining a plurality of characteristics of the micro-object within the hologram from the acquired digital holographic data. The plurality of characteristics of the micro-objects include a mean thickness value of the micro-objects, a coefficient of variation of the thickness of the micro-objects, a skewness value of the thickness of the micro-objects, a kurtosis value of the thickness of the micro-objects, a projected area of ​​the micro-objects, an optical volume of the micro-objects, etc. By applying a pre-trained classifier to the determined plurality of characteristics, it is determined whether the micro-objects belong to a particular type of micro-object.

[0004] Infertility treatments such as IVF and ICSI involve collecting pre-ovulatory eggs from a patient, fertilizing them outside the patient's body, and then transferring the resulting fertilized eggs (embryos) back into the uterus. When multiple embryos are obtained through IVF or ICSI, the embryos used for embryo transfer are determined by their grade. Currently, embryo grading is primarily based on the embryo's morphology.

[0005] If it were possible to predict the quality of embryos at the very early stage (fertilized egg pronuclear stage) before they develop into blastocysts, it would be possible to increase the success rate of infertility treatments. However, the morphology of embryos at the fertilized egg pronuclear stage is similar, making it difficult to predict quality based on the morphological characteristics of embryos. The technologies described in the above patent documents all utilize the morphological characteristics of cells and are not applicable to predicting the quality of embryos at the fertilized egg pronuclear stage. It would be beneficial to be able to predict the quality of not only embryos but also somatic cells (e.g., blood cells, muscle cells, skin cells, nerve cells, gland cells, and organ cells) and stem cells (e.g., iPS cells (induced pluripotent stem cells), ES cells (embryonic stem cells), and adult stem cells) without relying on morphological characteristics.

[0006] The disclosed technology has been made in consideration of the above points, and aims to enable quality prediction of cells without relying on morphological characteristics.

[0007] The information processing device includes at least one processor that acquires a phase image generated based on a hologram of a cell captured using digital holography technology, and derives evaluation information indicating the likelihood that the cell will change into a predetermined morphology based on distribution information indicating a distribution of phase values ​​in the phase image.

[0008] The processor may derive a histogram of phase values ​​of the phase image as the distribution information. The processor may derive evaluation information based on the frequency of each class in the histogram. The processor may derive the evaluation information using a trained model that receives the frequency of each class in the histogram as input and outputs evaluation information. The processor may derive a probability density distribution of phase values ​​for the phase image as the distribution information.

[0009] The processor may acquire a plurality of phase images generated based on a plurality of holograms obtained for the same cell at different times, derive distribution information for each of the plurality of phase images, and derive the final evaluation information by statistically processing the evaluation information derived based on each of the plurality of distribution information. The processor may generate the phase image based on the hologram of the cell. The processor may control capturing of the hologram of the cell.

[0010] The cell may be an embryo at the pronuclear stage of a fertilized egg, and the evaluation information may be the probability that the embryo will change into a blastocyst. The cell may be a cancer cell or a cancer stem cell, and the evaluation information may be the probability that the cancer cell or the cancer stem cell will change into a dead cell. The cell may be a stem cell, and the evaluation information may be the probability that an undifferentiated cell will change into a differentiated cell.

[0011] A cell evaluation system according to the disclosed technique includes the above-described information processing device, an optical system configured to form a hologram, and an imaging device for capturing an image of the hologram.

[0012] The information processing method according to the disclosed technology involves at least one processor included in an information processing device performing a process of acquiring a phase image generated based on a hologram of a cell photographed using digital holography technology, and deriving evaluation information indicating the possibility that the cell will change into a predetermined form based on distribution information indicating the distribution of phase values ​​in the phase image.

[0013] The program relating to the disclosed technology is a program for causing at least one processor provided in an information processing device to execute a process of acquiring a phase image generated based on a hologram of a cell photographed using digital holography technology, and deriving evaluation information indicating the possibility that the cell will change into a predetermined form based on distribution information indicating the distribution of phase values ​​in the phase image.

[0014] The learning method according to the disclosed technology uses a data set as learning data, which is a combination of distribution information indicating the distribution of phase values ​​in a phase image generated based on a hologram of a cell photographed using digital holography technology and information indicating whether the cell has changed into a predetermined morphology, and trains a model that inputs the distribution information and outputs evaluation information indicating the possibility that the cell will change into a predetermined morphology.

[0015] The disclosed technology makes it possible to predict the quality of cells independently of morphological characteristics.

[0016] 6A is a diagram showing an example of the configuration of a cell evaluation system according to an embodiment of the disclosed technology. FIG. 6B is a diagram showing an example of the configuration of a substrate for culturing cells according to an embodiment of the disclosed technology. FIG. 6C is a diagram showing an example of the configuration of a stage-top incubator according to an embodiment of the disclosed technology. FIG. 6D is a diagram showing an example of the hardware configuration of an information processing device according to an embodiment of the disclosed technology. FIG. 6E is a flowchart showing the flow of processing performed by an information processing device according to an embodiment of the disclosed technology. FIG. 6F is a diagram showing an example of a hologram of a mouse fertilized egg (embryo) captured by an imaging device according to an embodiment of the disclosed technology. FIG. 6G is a diagram showing an example of a phase image of a mouse fertilized egg (embryo) generated based on the hologram shown in FIG. 6A. FIG. 6H is a diagram showing the concept of a phase image. FIG. 6I is a functional block diagram showing an example of the functional configuration of an information processing device according to an embodiment of the disclosed technology when the information processing device predicts the quality of a cell. FIG. 6J is a diagram showing an example of a histogram of phase values ​​in a phase image. FIG. 6J is a diagram showing the input and output of a quality prediction model according to an embodiment of the disclosed technology. FIG. 6F is a phase image of a mouse embryo at the pronuclear stage of a fertilized egg that was able to develop into a blastocyst. FIG. 6F is a phase image of a mouse embryo at the pronuclear stage of a fertilized egg that was not able to develop into a blastocyst. FIG. 6F shows odds for each phase class in a trained quality prediction model according to an embodiment of the disclosed technology. 10 is a flowchart showing an example of a process flow performed by a CPU when the CPU executes a quality prediction program. 11 is a flowchart showing a continuous probability density distribution derived by kernel density estimation for the same phase image as the phase image that is the original data of the histogram shown in FIG.

[0017] Hereinafter, an example of an embodiment of the disclosed technology will be described with reference to the drawings. In each drawing, the same or equivalent components and parts are given the same reference numerals, and redundant description will be omitted.

[0018] 1 is a diagram showing an example of the configuration of a cell evaluation system 1 according to an embodiment of the disclosed technology. The cell evaluation system 1 has the functions of capturing a hologram of a cell using digital holography technology, generating a phase image of the cell from the hologram, and predicting the quality of the cell based on the phase image. The cell evaluation system 1 is configured to include a hologram optical system 200, an imaging device 40, and an information processing device 10.

[0019] The hologram optical system 200 is configured to form an interference image (hologram) due to interference between the object light L1 and the reference light L2, and includes a splitter 21, reflecting mirrors 22 and 24, an objective lens 23, an imaging lens 25, a combiner 26, a phase shifter 30, a rotation mechanism 31, and a stage 70. A cell 60 to be evaluated is placed between the reflecting mirror 22 and the objective lens 23 in a state where it is cultured on a substrate 61. In this embodiment, the cell 60 is a fertilized egg (embryo).

[0020] The substrate 61 may be in the form of a dish having a plurality of wells 62 at the bottom, as shown in Fig. 2, for example. A plurality of cells 60 are cultured in each of the wells 62. As shown in Fig. 3, the substrate 61 is housed in a culture space 81 of a stage-top incubator 80. The temperature in the culture space 81 is controlled by an upper glass heater 82 and a bottom glass heater 83. CO 2 adjusted to an appropriate concentration is introduced into the culture space 81. ２ Gas is introduced through port 84. Hologram photography of cells 60 is performed with substrate 61 housed in culture space 81 of stage-top incubator 80. Stage-top incubator 80 is placed on stage 70 of hologram optical system 200. Stage 70 is movable in the horizontal and vertical directions. This makes it possible to sequentially perform hologram photography for each of the multiple cells 60 housed in each well 62.

[0021] Laser light L0, which is linearly polarized light emitted from a laser light source 20, is split into two laser lights by a splitter 21. One of the two laser lights is designated as object light L1, and the other is designated as reference light L2. A beam splitter can be used as the splitter 21. The object light L1 passes through a phase shifter 30 held by a rotation mechanism 31 and is incident on a reflecting mirror 22. The object light L1, whose traveling direction has been bent by the reflecting mirror 22, is irradiated onto a cell 60 on a substrate 61.

[0022] An image formed by the object light L1 transmitted through the cell 60 and the substrate 61 is enlarged by the objective lens 23. The direction of travel of the object light L1 transmitted through the objective lens 23 is bent by the reflecting mirror 24, and the light is incident on the combiner 26 via the imaging lens 25. Meanwhile, the reference light L2 is also incident on the combiner 26. The object light L1 and the reference light L2 are combined by the combiner 26, and an image is formed on the imaging plane of the imaging device 40. A beam splitter can be used as the combiner 26.

[0023] An interference image (hologram) generated by the interference between the object light L1 and the reference light L2 is captured by the imaging device 40. The imaging device 40 includes an imaging element such as a complementary metal-oxide-semiconductor (CMOS) image sensor, and generates image data of the interference image. The image data of the interference image is transmitted to the information processing device 10.

[0024] The phase shifter 30 is disposed between the demultiplexer 21 and the reflecting mirror 22 on the optical path of the object light L1. The phase shifter 30 has the function of changing the polarization direction of the object light L1. That is, the phase shifter 30 changes the polarization direction of the incident light to a direction according to the rotation position of the phase shifter 30 and outputs the light. The phase shifter 30 may be, for example, a half-wave plate (λ / 2 plate). The rotation mechanism 31 rotates the phase shifter 30 around the optical axis of the object light L1 as the rotation axis based on a control signal supplied from the information processing device 10. By rotating the phase shifter 30, it is possible to set the polarization direction of the object light L1 transmitted through the phase shifter 30 to any direction.

[0025] The interference image (hologram) of the object light L1 and the reference light L2 is formed by combining the object light L1 and the reference light L2 that have passed through the cell 60 and the substrate 61. In order for the object light L1 and the reference light L2 to interfere with each other, the object light L1 and the reference light L2 must each be linearly polarized and have the same polarization direction (polarization axis). However, if the substrate 61 is made of a material that generates birefringence, birefringence occurs when the object light L1 passes through the substrate 61, and the object light L1 becomes elliptically polarized. This causes the polarization axis of the object light L1 to rotate, and the polarization direction of the object light L1 and the polarization direction of the reference light L2 become misaligned. As a result, the coherence between the object light L1 and the reference light L2 decreases, reducing the contrast of the interference image (hologram). However, by rotating the phase shifter 30 to change the polarization direction of the object light L1, it is possible to align the polarization direction of the object light L1 with the polarization direction of the reference light L2, thereby suppressing a decrease in the contrast of the interference image (hologram).

[0026] The information processing device 10 derives an evaluation value for the interference image (hologram) captured by the imaging device 40 and controls the rotation position of the rotation mechanism 31 (and the phase shifter 30) based on the evaluation value. The evaluation value is an index value indicating the contrast of the interference image and may be, for example, the standard deviation or variance of pixel values ​​in the image data of the interference image. The pixel value means a numerical value indicating the brightness of the pixel. The higher the coherence between the object light L1 and the reference light L2, the higher the contrast of the interference image and the larger the standard deviation or variance as the evaluation value. The evaluation value can also be said to be an index value indicating the coherence between the object light L1 and the reference light L2. The information processing device 10 controls the rotation position of the rotation mechanism 31 (and the phase shifter 30) so as to maximize the evaluation value. This makes it possible to obtain a high-contrast interference image (hologram).

[0027] 4 is a diagram showing an example of the hardware configuration of the information processing device 10. The information processing device 10 includes a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102, a non-volatile memory 103, an input device 104 including a keyboard, a mouse, a microphone, etc., a display 105, and an interface 106. These pieces of hardware are connected to a bus 108.

[0028] The display 105 may be a touch panel display. The information processing device 10 can control the rotation mechanism 31, the stage 70, and the imaging device 40 via the interface 106. The non-volatile memory 103 is a non-volatile storage medium such as a hard disk or flash memory. The non-volatile memory 103 stores an imaging control program 110, an image processing program 120, a quality prediction program 130, and a quality prediction model 140.

[0029] The RAM 102 is a work memory for the CPU 101 to execute processing. The CPU 101 loads each program stored in the nonvolatile memory 103 into the RAM 102 and executes processing in accordance with each program. The CPU 101 is an example of a "processor" in the technology disclosed herein.

[0030] 5 is a flowchart showing the flow of processing performed by the information processing device 10 as a result of sequentially executing the imaging control program 110, the image processing program 120, and the quality prediction program 130. In step S1, the information processing device 10 executes the imaging control program 110 to capture a hologram of the cell 60. In step S2, the information processing device 10 executes the image processing program to generate a phase image of the cell 60 based on the hologram of the cell 60 acquired in step S1. In step S3, the information processing device 10 executes the quality prediction program 130 to predict the quality of the cell 60 based on the phase image of the cell 60 acquired in step S2. Details of each process in steps S1 to S3 are described below.

[0031] First, we will explain the hologram capture (step S1) of cell 60. Hologram capture of cell 60 is performed using hologram optical system 200 (see FIG. 1). Information processing device 10 controls the rotational position of rotation mechanism 31 so that the standard deviation or variance of pixel values ​​in the image data of the interference image (hologram) captured by imaging device 40 is maximized, i.e., so that the contrast of the interference image (hologram) is maximized.

[0032] In hologram capture of the cells 60, time-lapse capture is performed on each of the cells 60 to be evaluated. That is, hologram capture is performed on each of the cells 60 at regular time intervals (e.g., 30-minute intervals) over a predetermined period (e.g., 5 days) from the start of cell 60 culture. The information processing device 10 controls the capture timing of the imaging device 40 and the movement of the stage 70. The movement control of the stage 70 involves moving the stage 70 in a plane intersecting the optical axis (X and Y directions) so that the cells 60 cultured in each of the multiple wells 62 are sequentially positioned on the optical axis of the holographic optical system 200, and moving the stage 70 in the optical axis direction (Z direction) so that the cells 60 are focused on. The information processing device 10 executes the capture control program 110 to control the rotation position of the rotation mechanism 31, the capture timing of the imaging device 40, and the movement of the stage 70. FIG. 6A shows an example of a hologram of a mouse fertilized egg (embryo) captured by the imaging device 40. The image data of the hologram is transmitted from the imaging device 40 to the information processing device 10 .

[0033] Next, we will explain the generation (step S2) of a phase image of the cell 60. The information processing device 10 trims the hologram of the cell 60 captured by the imaging device 40 to a size of, for example, 2048 × 2048, and then performs a two-dimensional Fourier transform.

[0034] Next, the position of the object light is identified by determining the amount of deviation of the object light from the direct light in the Fourier transform image, and the complex amplitude component of only the object light is extracted by frequency filtering using, for example, a mask with a circular opening with a radius of 250 pixels.

[0035] Next, for example, an angular spectrum method is applied to restore an image showing the phase of a cell at an arbitrary spatial position. Specifically, the angular spectrum U(fx, fy; 0) of the Fourier transform image of the wavefront u(x, y; 0) captured on the imaging surface of the imaging device 40 is obtained. Next, as shown in the following equation (1), the angular spectrum U(fx, fy; 0) is multiplied by a transfer function H(fx, fy; z) to reconstruct the wavefront at an arbitrary position z in the optical axis direction (z direction). Here, the transfer function H(fx, fy; z) is a frequency response function (Fourier transform of the impulse response function (Green's function)).

[0036] Next, as shown in the following equation (2), an inverse Fourier transform is performed on the wavefront U(fx, fy; z) at position z in the optical axis direction (z direction) to derive a solution u(x, y; z) at position z.

[0037] Next, as shown in the following equation (3), a phase image is generated by deriving the phase φ for u(x, y; z).

[0038] The phase in the phase difference image before unwrapping is convoluted to values ​​between 0 and 2π. Therefore, the final phase image can be obtained by applying a phase unwrapping (unwrapping) method such as Unweighted Least Squares or Flynn's Algorithm to join the parts greater than 2π. Note that many unwrapping methods have been proposed, and an appropriate one that does not cause phase mismatch can be selected as appropriate. Each of the above processes is performed by the information processing device 10 executing the image processing program 120.

[0039] 6B is a diagram showing an example of a phase image of a mouse fertilized egg (embryo) generated based on the hologram shown in FIG. 6A. Phase images are generated at each time point in time-lapse photography for cells 60 cultured in each of a plurality of wells 62. Each of the generated phase images is stored in non-volatile memory 103.

[0040] The phase image will be described below. FIG. 7 shows the phase image I Ｐ The lower part of FIG. 7 shows the concept of the phase image I Ｐ The upper part of Fig. 7 shows a three-dimensional representation of the phase value at each pixel k of the phase image I Ｐ The phase value at each pixel k is shown in gray scale on a plane. Note that the phase value at each pixel k is the pixel value of that pixel.

[0041] Here, the phase image I Ｐ The phase of the background (area without cells) present in the same focal plane is P Ｂ and the phase of the region where the cells exist is P Ｓ Then, the phase image I Ｐ The phase value P in is expressed by the following equation (4): In addition, the term "phase" in this specification refers to the phase of the electric field amplitude when light is considered to be an electromagnetic wave, and is used in a more general sense.

[0042] Phase Image I Ｐ The phase value P at each pixel k ｋ can be expressed by the following formula (5): ｋ is the phase image I Ｐ is the refractive index of the cell at the site corresponding to each pixel k, and d ｋ is the phase image I Ｐ is the thickness of the cell at the site corresponding to each pixel k, and λ is the wavelength of the object light L1 in the hologram optical system 200.

[0043] The phase image of the cell 60 is an image showing the optical path length distribution of the object light L1 that has passed through the cell 60. Since the optical path length within the cell 60 corresponds to the product of the refractive index of the cell 60 and the thickness of the cell 60, the phase image of the cell 60 contains information on the refractive index and thickness (shape) of the cell 60, as shown in equation (5).

[0044] Next, quality prediction of the cell 60 (step S3) will be described. Fig. 8 is a functional block diagram showing an example of the functional configuration of the information processing device 10 when the information processing device 10 performs quality prediction of the cell 60. When the CPU 101 executes the quality prediction program 130, the information processing device 10 functions as a phase image acquisition unit 11, a phase distribution derivation unit 12, and a quality prediction unit 13.

[0045] The phase image acquisition unit 11 acquires multiple phase images of the cell 60 to be evaluated at the initial time point of the time-lapse imaging by reading them from the non-volatile memory 103. For example, the phase image acquisition unit 11 acquires the first four phase images of one cell 60, excluding the initial time point of the time-lapse imaging. These four phase images may be taken, for example, at 30 minutes, 60 minutes, 90 minutes, and 120 minutes after the start of culture. These four phase images are all phase images of an embryo at the pronuclear stage of a fertilized egg. It is preferable to exclude the cell 60 at the initial time point of the time-lapse imaging (i.e., immediately after the start of culture) from the processing target because it is expected that the cell 60 is in an unstable state immediately after thawing.

[0046] The phase distribution deriving unit 12 derives distribution information indicating the distribution of phase values ​​for each of the multiple phase images acquired by the phase image acquiring unit 11. The distribution information is information indicating the frequency with which each phase value is included in the phase image, or information indicating the relative relationship between the frequency of a certain phase value in the phase image and other phase values. In this embodiment, the distribution information is a histogram of phase values ​​in the phase image.

[0047] FIG. 9 is a diagram showing an example of a histogram of phase values ​​in a phase image. The histogram shown in FIG. 9 shows the number of pixels (frequency) belonging to each phase value class from 4 rad to 14.5 rad. In the histogram shown in FIG. 9, the class width is 0.5 rad, and the number of classes is 22. The phase distribution deriving unit 12 derives the number of pixels (frequency) in each class as distribution information. The distribution information is derived for each of the four phase images for each cell to be evaluated, which have different elapsed times from the start of culture.

[0048] The quality prediction unit 13 derives evaluation information indicating the possibility that the cell will change into a predetermined morphology based on the distribution information derived by the phase distribution derivation unit 12. In this embodiment, the quality prediction unit 13 derives the probability that an embryo at the pronuclear stage of a fertilized egg will change into a blastocyst as the evaluation information.

[0049] The quality prediction unit 13 derives evaluation information using a quality prediction model 140 that has been trained by machine learning. The quality prediction model 140 is a logistic regression model that receives distribution information (frequency of each class in the histogram of the phase image) for the cell and outputs evaluation information (the probability that the cell will develop into a blastocyst). As shown in FIG. 10 , the quality prediction model 140 receives four sets of distribution information (distribution information #1, #2, #3, and #4) derived for each of four different phase images for one cell 60. The quality prediction model 140 derives the probability that the embryo will develop into a blastocyst for each of the distribution information #1, #2, #3, and #4, and outputs the median of these probabilities as final evaluation information.

[0050] The quality prediction model 140 is trained using, as training data, a dataset consisting of a combination of distribution information (frequency of each class in a histogram of a phase image) for a certain cell and information (correct label) indicating whether the cell has changed to a predetermined morphology. Training the quality prediction model 140 is equivalent to constructing a logistic regression model. That is, training the quality prediction model 140 determines partial regression coefficients (weights) for a regression equation in which the distribution information is the explanatory variable and the evaluation information is the target variable. It is preferable to perform L1 regularization in training the quality prediction model 140. This makes it possible to prevent overtraining of the quality prediction model 140.

[0051] The relationship between distribution information (frequency of each class in the histogram of the phase image) and evaluation information (probability of the embryo developing into a blastocyst) will be described below. FIG. 11A is a phase image of a mouse embryo at the pronuclear stage of a fertilized egg that was able to develop into a blastocyst, and FIG. 11B is a phase image of a mouse embryo at the pronuclear stage of a fertilized egg that was not able to develop into a blastocyst. The phase image of the embryo that was able to develop into a blastocyst has a generally uniform phase value (brightness) from the center to the periphery of the embryo. On the other hand, the phase image of the embryo that was not able to develop into a blastocyst tends to have a relatively large difference in phase value (brightness) between the center and periphery of the fertilized embryo. Specifically, the phase value at the center of the embryo is relatively large, while the phase value at the periphery of the embryo is relatively small, and this difference is significant. The above-mentioned difference in phase image characteristics between embryos that were able to develop into blastocysts (hereinafter referred to as OK embryos) and embryos that were not able to develop into blastocysts (hereinafter referred to as NG embryos) is reflected in the histogram of the phase image. The quality prediction model 140 derives, as evaluation information, the probability that the embryo will develop into a blastocyst from a histogram that reflects the difference in phase image features between a good embryo and a bad embryo.

[0052] FIG. 12 shows the odds (odds) for each phase class in the trained quality prediction model 140 (logistic regression model). The odds are expressed by the following equation (6). In equation (6), p is the probability that the fertilized egg (embryo) in question will be able to develop into a blastocyst, and 1-p is the probability that the fertilized egg (embryo) in question will not be able to develop into a blastocyst. The odds obtained by dividing p by 1-p is an index that can evaluate the likelihood that the fertilized egg (embryo) in question will be able to develop into a blastocyst with higher sensitivity than the probability p itself.

[0053] Here, in the quality prediction model 140, a sigmoid function expressed by the following equation (7) is applied as a model function: In equation (7), x is a topological class value, and θ is a weight in the topological class value.

[0054] In the histogram according to this embodiment (see FIG. 9), the number of classes is 22, so the number of x and θ is 22. For ease of understanding, it is assumed here that the number of classes is two, and the phase class value x １ The weight for １ and the topological class value x ２ The weight for ２ That is, x = (x １ , x ２ ) and θ = (θ １ ,θ ２ In this case, the model function is expressed by the following equation (8). By training the quality prediction model 140, １ , θ ２ is determined. The f output from the trained quality prediction model 140 is θ (x) represents the probability that the embryo will be able to develop into a blastocyst.

[0055] p in equation (6) and f in equation (8) θ Substituting (x), the following equation (9) is obtained.

[0056] Here, all pixels in the phase image have a phase class value x１ and all pixels of the phase image have only phase values ​​belonging to the phase class value x ２ Consider the likelihood that the embryo can develop into a blastocyst if it has only phase values ​​belonging to the former case (x １ , x ２ ) = (1, 0), and in the latter case (x １ , x ２ )=(0,1), and the odds are calculated, the following equation (10) is obtained.

[0057] In Figure 12, exp(θ)-1 corresponds to equation (10) and represents the likelihood that the embryo will be able to develop into a blastocyst when all pixels in the phase image have only phase values ​​that belong to that phase class value. Note that in Figure 12, -1 is added to shift the reference to zero. On the other hand, exp(-θ)-1 in Figure 12 represents the likelihood that the embryo will not be able to develop into a blastocyst when all pixels in the phase image have only phase values ​​that belong to that phase class value.

[0058] 12 , for a phase image having a relatively large number of pixels belonging to the class values ​​11 rad and 13 rad, the quality prediction model 140 outputs a relatively high value as the probability that the embryo will develop into a blastocyst. Also, for a phase image having a relatively large number of pixels belonging to the class values ​​9 rad and 14 rad, the quality prediction model 140 outputs a relatively low value as the probability that the embryo will develop into a blastocyst. In fact, it has been confirmed that phase images of OK embryos tend to have a relatively large number of pixels belonging to the class values ​​11 rad and 13 rad, while phase images of NG embryos tend to have a relatively large number of pixels belonging to the class values ​​9 rad and 14 rad. The information processing device 10 according to this embodiment can predict the quality of the cell 60 (predicting whether an embryo at the pronuclear stage will develop into a blastocyst) with an accuracy of about 70%.

[0059] 13 is a flowchart showing an example of the flow of processing performed by the CPU 101 by executing the quality prediction program 130. The quality prediction program 130 is executed after the acquisition of phase images for each of the cells 60 to be evaluated in accordance with the image processing program 120 is completed.

[0060] In step S11, the CPU 101 functions as the phase image acquisition unit 11 and acquires a plurality of phase images of the evaluation target cell 60 at an initial time point of the time lapse photography by reading them from the non-volatile memory 103. For example, the CPU 101 acquires the first four phase images excluding the earliest time point of the time lapse photography for one cell 60. These four phase images are all phase images of an embryo at the pronuclear stage of a fertilized egg.

[0061] In step S12, the CPU 101 functions as the phase distribution deriving unit 12, and derives distribution information indicating the distribution of phase values ​​for each of the phase images acquired in step S11. In this embodiment, the distribution information is a histogram of phase values ​​in the phase images.

[0062] In step S13, the CPU 101 functions as the quality prediction unit 13 and derives evaluation information indicating the possibility that the cell 60 will change into a predetermined morphology based on the distribution information derived in step S12. In this embodiment, the CPU 101 derives the probability that an embryo at the pronuclear stage of a fertilized egg will change into a blastocyst as the evaluation information. The CPU 101 derives the evaluation information using a quality prediction model 140 that has been trained by machine learning. As shown in FIG. 10 , the quality prediction model 140 receives four sets of distribution information (distribution information #1, #2, #3, and #4) derived for each of four different phase images for one cell 60. The quality prediction model 140 derives the probability that the embryo will change into a blastocyst for each of the distribution information #1, #2, #3, and #4, and outputs the median of these probabilities as the final evaluation information.

[0063] As described above, the information processing device 10 according to this embodiment acquires a phase image generated based on a hologram of a cell 60 photographed using digital holography technology, and derives evaluation information indicating the possibility that the cell 60 will change into a predetermined form based on distribution information indicating the distribution of phase values ​​in the phase image.

[0064] The morphology of embryos at the pronuclear stage of a fertilized egg is similar, making it difficult to predict quality based on the morphological characteristics of the embryo. The information processing device 10 according to this embodiment makes it possible to predict the quality of embryos at a stage where there are no substantial differences in morphological characteristics. Being able to predict the quality of embryos at the very early stage can contribute to improving the success rate of infertility treatment.

[0065] Here, we consider the case where bright-field images obtained using an optical microscope or the like are used instead of phase images. Bright-field images can vary in brightness and color depending on the sample placement, light intensity, and angle of incidence, making it difficult to obtain quantitative data on features other than morphological features. Therefore, the histogram of a bright-field image merely becomes noise when using it as a feature to predict cell quality. This also applies when using a phase-contrast microscope image or a differential interference contrast image. On the other hand, the phase value of each pixel in a phase image has physical meaning and is reproducible. Therefore, the histogram of a phase image functions effectively when using it as a feature to predict cell quality.

[0066] In the above description, an example has been given in which evaluation information is derived based on four phase images for one cell. However, evaluation information may be derived based on fewer than four phase images or five or more phase images. Evaluation information may also be derived based on a single phase image. By deriving the probability that the embryo will develop into a blastocyst for each of multiple phase images and using the median of these probabilities as the final evaluation information, it is possible to improve the accuracy of cell quality prediction. The multiple phase images may be any combination as long as they are from the pronuclear stage of a fertilized egg.

[0067] Furthermore, although the above description illustrates a case where evaluation information is derived using a single quality prediction model, the quality prediction model may be constructed as an ensemble model. For example, a plurality of (e.g., five) models closest to the median balanced accuracy obtained during training through leave-one-out cross-validation (LOOCV) may be selected, and the median of the "probability of the embryo developing into a blastocyst" calculated by each selected model may be used as the final evaluation information. In this case, each model may be caused to derive the "probability of the embryo developing into a blastocyst" for each of a plurality of mutually different phase images, and the median of the calculated values ​​may be used as the output value of each model, and the median of the output values ​​of each model may be used as the final evaluation information.

[0068] Furthermore, although the above description has been given as an example of deriving evaluation information for mouse embryos, the disclosed technology can also be applied to embryos of mammals other than mice.

[0069] In the above description, a histogram of phase values ​​is used as distribution information showing the distribution of phase values ​​in a phase image, but the disclosed technology is not limited to this. A histogram shows a discrete phase distribution in which the frequency of phase values ​​is divided into classes. The distribution of phase values ​​in a phase image is not limited to this, and can also be expressed as a continuous probability density distribution using a non-parametric method such as kernel density estimation (KDE). Therefore, a phase distribution expressed as a continuous probability density distribution can be used as distribution information.

[0070] Fig. 14 shows a continuous probability density distribution derived by kernel density estimation for the same phase image as the phase image that is the original data for the histogram shown in Fig. 9. The probability density distribution shown in Fig. 14 uses a Gaussian kernel and is calculated using the following equation (11).

[0071] In equation (11), x is the phase [rad], n is the number of data, and X ｉis the phase value for each pixel in the phase image (1024 x 1024). When extracting data in a range of 4 rad or more, the number of data (number of pixels) n is approximately 100,000. h is the bandwidth of each Gaussian kernel, and in the above example, h = 1.0. There are various algorithms for determining the appropriate bandwidth.

[0072] When predicting cell quality using a histogram of phase values, narrowing the class width to increase the number of feature dimensions (number of classes) to improve prediction accuracy results in a noisy profile. Conversely, widening the class width to decrease the number of feature dimensions (number of classes) tends to blur the characteristics of the phase distribution. There are various algorithms for determining an appropriate class width, but in any case, it is not possible to increase the number of feature dimensions (number of classes) without limit.

[0073] On the other hand, a continuous probability density distribution allows values ​​to be evaluated at any point. In general, increasing the number of dimensions improves the expressive power of a quality prediction model, thereby increasing prediction accuracy. It is effective to apply a continuous probability density distribution as distribution information instead of a histogram of phase values.

[0074] Furthermore, the above description has been given as an example of deriving evaluation information for cells in the early stages of morphological change (embryos in the pronuclear stage of a fertilized egg), but the disclosed technology is not limited to this aspect. Evaluation information can also be derived for cells in the middle and late stages of morphological change. For cells in the middle and late stages of morphological change, other evaluation methods (e.g., quality prediction based on morphological characteristics of cells) may be applicable. In this case, the other evaluation methods may be combined with the evaluation method according to the disclosed technology. For example, evaluation information may be derived by taking into account index values ​​based on morphological characteristics of cells.

[0075] Furthermore, in the above description, an example has been given in which the probability that an embryo at the pronuclear stage of a fertilized egg will develop into a blastocyst is derived as evaluation information, but the disclosed technology is not limited to this aspect. The disclosed technology can also be applied to, for example, a case in which information indicating the likelihood that a fertilized egg (embryo) at an early, mid, or late stage will implant is derived as evaluation information. In this aspect, if other evaluation methods are applicable, the other evaluation methods may be combined with the evaluation method according to the disclosed technology. For example, the evaluation information may be derived by taking into account index values ​​based on the morphological characteristics of the cells.

[0076] Furthermore, while the above description exemplifies a case in which the target cell is a fertilized egg (embryo), the disclosed technology is not limited to this aspect. The disclosed technology can also be applied, for example, to deriving evaluation information about stem cells. Stem cells include, for example, iPS cells (induced pluripotent stem cells), ES cells (embryonic stem cells), and adult stem cells. Adult stem cells include, for example, hematopoietic stem cells, intestinal stem cells, neural stem cells, skin stem cells, and mammary stem cells. For example, the disclosed technology can be applied to the morphological change from a differentiated cell (e.g., a human cell) to an undifferentiated cell (e.g., an iPS cell). That is, the evaluation information may be information indicating the possibility that a differentiated cell (e.g., a human cell) will transform into an undifferentiated cell (e.g., an iPS cell). The disclosed technology can also be applied to the morphological change from an undifferentiated cell (e.g., an iPS cell) to a differentiated cell (e.g., a human cell). That is, the evaluation information may be information indicating the possibility that an undifferentiated cell (e.g., an iPS cell) will transform into a differentiated cell (e.g., a human cell). The disclosed technology can also be applied to the morphological change from a normal stem cell to a cancer cell or a cancer stem cell. That is, the evaluation information may be information indicating the possibility that normal stem cells will change into cancer cells or cancer stem cells. According to this aspect, the evaluation information can be used, for example, as an index for evaluating drug efficacy. Furthermore, the disclosed technology can be applied to the morphological change from cancer cells or cancer stem cells to dead cells. That is, the evaluation information may be information indicating the possibility that cancer cells or cancer stem cells will change into dead cells. According to this aspect, the evaluation information can be used, for example, as an index for evaluating drug efficacy. In these aspects, if other evaluation methods are applicable, the other evaluation methods may be combined with the evaluation method according to the disclosed technology. For example, the evaluation information may be derived by taking into account an index value based on the morphological characteristics of the cells.

[0077] The disclosed technology can also be applied, for example, to the case of deriving evaluation information for somatic cells. Somatic cells include, for example, blood cells, muscle cells, skin cells, nerve cells, gland cells, and organ cells. Skin cells include, for example, epidermal cells and dermal cells. Organ cells include, for example, cardiac muscle cells, lung cells, intestinal cells, liver cells, kidney cells, and brain cells. For example, the disclosed technology can be applied to the morphological change of normal cells into cancer cells or cancer stem cells. That is, the evaluation information may be information indicating the possibility that normal cells will change into cancer cells or cancer stem cells. According to this aspect, the evaluation information can be used, for example, as an index for evaluating drug efficacy. The disclosed technology can also be applied to the morphological change of cancer cells or cancer stem cells into dead cells. That is, the evaluation information may be information indicating the possibility that cancer cells or cancer stem cells will change into dead cells. According to this aspect, the evaluation information can be used, for example, as an index for evaluating drug efficacy. In these aspects, if other evaluation methods are applicable, the other evaluation methods may be combined with the evaluation method according to the disclosed technology. For example, evaluation information may be derived by taking into account index values ​​based on the morphological characteristics of the cells.

[0078] The cells that are the subject of the disclosed technology are not limited to single cells, but may be multicellular. Furthermore, the cells may form colonies, spheroids, or tissues. Furthermore, the cells may be organoids. Furthermore, while the above description focuses on cells arranged on a substrate, the subject cells may also be arranged in a flow channel or a microphysiological system (MPS).

[0079] The following supplementary notes are further disclosed regarding the above-described embodiments: (Supplementary Note 1) An information processing device having at least one processor, wherein the processor acquires a phase image generated based on a hologram of a cell captured using digital holography technology, and derives evaluation information indicating a possibility that the cell will change into a predetermined morphology based on distribution information indicating a distribution of phase values ​​in the phase image.

[0080] (Supplementary Note 2) The information processing device according to Supplementary Note 1, wherein the processor derives a histogram of phase values ​​of the phase image as the distribution information.

[0081] (Supplementary Note 3) The information processing device according to Supplementary Note 2, wherein the processor derives the evaluation information based on a frequency of each class in the histogram.

[0082] (Supplementary Note 4) The information processing device according to Supplementary Note 3, wherein the processor derives the evaluation information using a trained model that receives the frequency of each class in the histogram as an input and outputs the evaluation information.

[0083] (Supplementary Note 5) The information processing device according to Supplementary Note 1, wherein the processor derives a probability density distribution of phase values ​​for the phase image as the distribution information.

[0084] (Supplementary Note 6) The information processing device described in any one of Supplementary Note 1 to Supplementary Note 5, wherein the processor acquires a plurality of phase images generated based on a plurality of holograms obtained for the same cell at different times, derives the distribution information for each of the plurality of phase images, and derives the evaluation information derived based on each of the plurality of distribution information as final evaluation information.

[0085] (Supplementary Note 7) The information processing device according to any one of Supplementary Notes 1 to 6, wherein the processor generates the phase image based on a hologram of a cell.

[0086] (Supplementary Note 8) The information processing device according to any one of Supplementary Notes 1 to 7, wherein the processor controls capturing a hologram of the cell.

[0087] (Supplementary Note 9) The information processing device according to any one of Supplementary Note 1 to Supplementary Note 8, wherein the cell is an embryo in a pronuclear stage of a fertilized egg, and the evaluation information is a probability that the embryo will develop into a blastocyst.

[0088] (Supplementary Note 10) The information processing device according to any one of Supplementary Note 1 to Supplementary Note 8, wherein the cells are cancer cells or cancer stem cells, and the evaluation information is a probability that the cancer cells or cancer stem cells will change into dead cells.

[0089] (Supplementary Note 11) The information processing device according to any one of Supplementary Note 1 to Supplementary Note 8, wherein the cells are stem cells, and the evaluation information is a probability that an undifferentiated cell will change into a differentiated cell.

[0090] (Supplementary Note 12) A cell evaluation system comprising: an information processing device according to any one of Supplementary Notes 1 to 11; an optical system configured to form the hologram; and an imaging device for photographing the hologram.

[0091] (Supplementary Note 13) An information processing method in which at least one processor included in an information processing device executes the following processes: acquiring a phase image generated based on a hologram of a cell photographed using digital holography technology; and deriving evaluation information indicating the possibility that the cell will change into a predetermined morphology based on distribution information indicating the distribution of phase values ​​in the phase image.

[0092] (Supplementary Note 14) A program for causing at least one processor provided in an information processing device to execute the following process: acquiring a phase image generated based on a hologram of a cell photographed using digital holography technology; and deriving evaluation information indicating the possibility that the cell will change into a predetermined morphology based on distribution information indicating the distribution of phase values ​​in the phase image.

[0093] (Supplementary Note 15) A learning method using, as training data, a dataset consisting of a combination of distribution information indicating the distribution of phase values ​​in a phase image generated based on a hologram of a cell photographed using digital holography technology and information indicating whether the cell has changed into a predetermined morphology, and training a model that uses the distribution information as input and outputs evaluation information indicating the possibility that the cell will change into the predetermined morphology.

[0094] The disclosure of Japanese Patent Application No. 2023-120319, filed on July 24, 2023, is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually indicated to be incorporated by reference.

Claims

1. An information processing device having at least one processor, wherein the processor acquires a phase image generated based on a hologram of a cell captured using digital holography technology, and derives evaluation information indicating the possibility that the cell will change into a predetermined morphology based on distribution information indicating the distribution of phase values ​​in the phase image.

2. The information processing device according to claim 1, wherein the processor derives a histogram of phase values ​​of the phase image as the distribution information.

3. The information processing device according to claim 2, wherein the processor derives the evaluation information based on the frequency of each class in the histogram.

4. The information processing device according to claim 3, wherein the processor derives the evaluation information using a trained model that receives the frequency of each class in the histogram as an input and outputs the evaluation information.

5. The information processing device according to claim 1, wherein the processor derives a probability density distribution of phase values ​​for the phase image as the distribution information.

6. The information processing device according to claim 1, wherein the processor acquires a plurality of phase images generated based on a plurality of holograms obtained for the same cell at different times, derives the distribution information for each of the plurality of phase images, and derives the final evaluation information by statistically processing the evaluation information derived based on each of the plurality of distribution information.

7. The information processing device according to claim 1, wherein the processor generates the phase image based on a hologram of a cell.

8. The information processing device according to claim 1, wherein the processor controls the capturing of a hologram of the cell.

9. The information processing device according to claim 1, wherein the cell is an embryo in the pronuclear stage of a fertilized egg, and the evaluation information is the probability that the embryo will develop into a blastocyst.

10. The information processing device according to claim 1, wherein the cells are cancer cells or cancer stem cells, and the evaluation information is the probability that the cancer cells or cancer stem cells will change into dead cells.

11. The information processing device according to claim 1, wherein the cells are stem cells, and the evaluation information is the probability that an undifferentiated cell will change into a differentiated cell.

12. A cell evaluation system comprising: an information processing device according to any one of claims 1 to 11; an optical system configured to form the hologram; and an imaging device for photographing the hologram.

13. An information processing method in which at least one processor provided in an information processing device performs the following processes: acquiring a phase image generated based on a hologram of a cell photographed using digital holography technology; and deriving evaluation information indicating the possibility that the cell will change into a specified morphology based on distribution information indicating the distribution of phase values ​​in the phase image.

14. A program for causing at least one processor provided in an information processing device to execute the following process: acquiring a phase image generated based on a hologram of a cell photographed using digital holography technology; and deriving evaluation information indicating the possibility that the cell will change into a predetermined morphology based on distribution information indicating the distribution of phase values ​​in the phase image.

15. A learning method using, as learning data, a dataset consisting of a combination of distribution information indicating the distribution of phase values ​​in a phase image generated based on a hologram of a cell photographed using digital holography technology and information indicating whether the cell has changed into a specified morphology, and training a model that uses the distribution information as input and outputs evaluation information indicating the possibility that the cell will change into a specified morphology.

Images