Cell observation image capturing device and cell observation image acquiring method

Abstract

To secure continuity of observation images, and shorten a required imaging time.SOLUTION: An imaging apparatus of cell observation image that mounts a culture vessel carrying a cell to be cultured thereon, and captures observation images of a cell, and comprises: an illumination unit which irradiates a culture vessel with illumination light; an imaging unit which images the culture vessel irradiated by the illumination light; a computer which executes various calculations; a storage part which stores calculation results by a computer; and an output part which outputs the calculation results, in which a predetermined contrast pattern is arranged between the illumination unit and a top lid of the culture vessel.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a cell observation image capturing device and a cell observation image acquiring method. [Background technology]

[0002] With the development of regenerative medicine technology, there are an increasing number of cases where cells are cultivated industrially. Unlike reactions of chemical substances, the properties of cultured cells vary widely and are easily affected by factors such as the environment. For this reason, there is an increasing need to monitor the state of cell culture. In this regard, in the current field of regenerative medicine, cells are observed using microscopes.

[0003] Generally, microscopes have a very narrow field of view, so only a part of the cell culture vessel can be observed at one time. Therefore, in order to obtain information on the entire surface of the vessel, a technique called tiling photography is used to increase the field of view by moving the stage on which the cell culture vessel is placed using an electric XY stage, or by moving the camera and lighting set, and then stitching together the images taken while positioning the vessel to generate an observation image of the entire surface of the vessel. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2011-127972 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, tiling photography has problems such as differences in brightness between adjacent images, differences in the photography location between adjacent images due to limitations in the positional accuracy of the motorized stage, and even inconsistencies between adjacent images caused by cells moving during photography, which results in discontinuity within the images. It is desirable to automate the monitoring of the culture status using a computer, but images with internal discontinuities are not compatible with automatic processing.

[0006] Another issue with tiling photography is that when you want to capture a wide area, the number of repetitions of stage movement and photography increases, which means the time required for photography is long. Unless special consideration is given, the environment for photography is often different from that suitable for cell culture. If cells are exposed to an environment that is not suitable for culture for a long period of time, it will have a negative effect on the culture itself. In addition, if the time required is long, the frequency of monitoring will decrease, making it difficult to respond promptly. In view of such circumstances, the present disclosure proposes a technique for ensuring the continuity of observed images and shortening the time required for capturing images. [Means for solving the problem]

[0007] In order to solve the above problems, according to the present embodiment, there is provided a cell observation image capturing device for placing a culture vessel carrying cells to be cultured and capturing an observation image of the cells, the cell observation image capturing device comprising: an illumination unit for irradiating illumination light onto the culture vessel; and a photographing unit for photographing the culture vessel illuminated by the illumination light, wherein a predetermined pattern having a difference in light transmittance is arranged between the illumination unit and the top lid of the culture vessel; and the focus of the photographing unit is adjusted to a position shifted a predetermined distance toward the illumination unit from the culture vessel placement surface of the photographing unit.

[0008] Further features related to the present disclosure will be apparent from the description of the present specification and the accompanying drawings. Also, aspects of the present disclosure may be realized and realized by the elements and combinations of various elements and aspects set forth in the following detailed description and the appended claims. It should be understood that the descriptions in this specification are exemplary and illustrative only and are not intended to limit the scope or application of the present disclosure in any way. Effect of the Invention

[0009] According to the present disclosure, it is possible to ensure the continuity of observed images and reduce the time required for imaging. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram showing an example of the configuration of a scanner device 100 for capturing a cell observation image according to the present embodiment. [Diagram 2] This is a diagram for comparing an image when the cell 1102 is in focus (Figure 2A) with an image when the cell 1102 is out of focus (Figure 2B: when the focus is on the glass surface of the photographing unit, as with a normal scanner). [Diagram 3] 1A to 1C are diagrams showing an example of a pattern configuration of a pattern sheet 1011 used in the scanner device 100 for capturing cell observation images according to this embodiment. [Figure 4] FIG. 13 is a diagram for explaining the principle by which cells can be photographed by adding a pattern sheet. [Diagram 5] FIG. 5A shows an image observed by microscope tiling photography, FIG. 5B shows an image observed by conventional scanner photography, and FIG. 5C shows an image observed by scanner photography of the present invention. [Figure 6] 1A and 1B are diagrams showing examples of an image acquired by the scanner device 100 for capturing cell observation images and an image paired with the image acquired by the scanner device 100. FIG. [Figure 7] FIG. 1 is a diagram illustrating the concept of machine learning using paired images. [Figure 8] FIG. 13 is a diagram showing the concept of inputting an actual observation image with a pattern sheet background and outputting an observation image without the background. [Figure 9] 11 is a diagram for explaining the operation of the scanner device 200 for capturing cell observation images according to the second embodiment. FIG. [Figure 10]13 is a diagram for explaining the operation of the scanner device 300 for capturing cell observation images according to the third embodiment. FIG. [Figure 11] FIG. 13 is a diagram for explaining the operation of the scanner device 400 for capturing cell observation images according to the fourth embodiment. [Figure 12] FIG. 13 is a diagram showing an example of the configuration of a pattern sheet according to a fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In the accompanying drawings, functionally identical elements may be indicated by the same numbers. Note that the accompanying drawings show specific embodiments and implementation examples according to the principles of the present disclosure, but these are for understanding the present disclosure and are by no means used to interpret the present disclosure in a limiting manner.

[0012] In the present embodiment, the description is given in sufficient detail for a person skilled in the art to implement the present disclosure, but it should be understood that other implementations and forms are possible, and that changes in configuration and structure and substitutions of various elements are possible without departing from the scope and spirit of the technical ideas of the present disclosure. Therefore, the following description should not be interpreted as being limited thereto.

[0013] This embodiment proposes a scanner suitable for acquiring observation images of cells in order to ensure the continuity of the observation images and to shorten the time required for imaging.

[0014] Due to fundamentally significant issues with scanner photography, no attempts have been made to use a scanner to photograph culture vessels and obtain observation images. This is because normal scanners cannot photograph highly transparent objects such as cells. For example, if a culture vessel (dish) during cell culture is photographed with a scanner in the conventional manner, the entire image will be almost completely white, with almost no cells visible. This is the same for both reflective and transmissive scanners. Note that with microscopes, ingenuity has been employed to observe cells, such as using the "wave properties of light" rather than "color (absorption at each wavelength)" (phase contrast microscope), but scanners have not been used in this way because their original purpose is different from that of microscopes. Also, in principle, microscopes cannot photograph objects that are highly transparent, so the images are not captured by scanners. shadow A surface is a two-dimensional optical structure, whereas a scanner is a shadow The surface has a one-dimensional optical structure. Furthermore, there is currently no research into whether the same principle as a phase contrast microscope can be applied to a scanner.

[0015] Therefore, this embodiment proposes a method for acquiring cell observation images as an alternative to conventional tiling photography by enabling "cell photography with a scanner," which was previously impossible, thereby ensuring continuity of the observation images and shortening the time required for photography.

[0016] (1) First embodiment <Example of the configuration of a scanner for capturing images of cells> FIG. 1 is a diagram showing an example of the configuration of a scanner device 100 for capturing a cell observation image according to this embodiment.

[0017] The scanner device 100 for photographing cell observation images is, for example, a transmissive scanner device, and includes an illumination unit 101 and an imaging unit 102 that are provided in a normal scanner, a computer 103 that controls the illumination unit 101 and the imaging unit (including a scanner head) 102, an input device (mouse, keyboard, etc.) 104 for, for example, a user to input commands and parameters, a storage device 105 for, for example, storing photographed observation images, an output device (display device or printer) 106 for, for example, outputting (displaying, printing) the photographed observation images, and an illumination unit 101 and an imaging unit 102 controlled by the computer 103. shadow and a drive unit (not shown) that translates the unit 102 in a synchronous and interlocking manner.

[0018] The illumination unit 101 is attached to, for example, the cover of the scanner, in the same manner as in a normal scanner. A pattern sheet 1011 is attached to the surface (contact glass surface) of the illumination unit 101 facing the cell culture vessel 110 (alternatively, a pattern may be printed (glass printing) on ​​the glass surface, or a light and dark pattern may be formed by processing the glass surface such as unevenness). Alternatively, when taking a cell observation image, the pattern sheet 1011 independent of the scanner (separate: not attached to the glass surface of the illumination unit 101) may be placed on the top cover of the cell culture vessel 110. Alternatively, the pattern sheet 1011 may be fixed in some manner between the top cover of the cell culture vessel 110 and the illumination unit 101. In this way, a distance of a predetermined value or more can be provided between the cells to be photographed and the pattern sheet 1011, and the cells will not be blocked by the pattern. The pattern of the pattern sheet 1011 will be described later.

[0019] Unlike a normal scanner (where the scanner head is focused on the glass surface of the imaging unit on which the object to be imaged is placed), the imaging unit 102 has the scanner head focused upward by a predetermined distance (for example, at least the thickness of the bottom glass of the cell culture vessel 110). shadowThe focus of the unit is shadow The cell observation image capturing device 100 is configured to be a refractive surface of the cell (cell surface: meaning including the case where the cell is attached to the bottom surface of the culture vessel 110 or the case where the cell is slightly raised from the bottom surface). The position of the cell 1102 to be observed is the bottom surface of the cell culture vessel 110, but this position is about 1 mm above the glass surface of the photographing unit 102. FIG. 2 is a diagram for comparing an image when the cell 1102 is in focus (FIG. 2A) with an image when the cell 1102 is not in focus (FIG. 2B, the glass surface of the photographing unit is in focus like a normal scanner). As can be seen from FIG. 2, when the cell 1102 is photographed by the cell observation image capturing scanner device 100 according to this embodiment in which the cell 1102 is in focus, an observation image in which each cell 1102 is very clearly reflected can be obtained. The focus of the scanner head may be fixed, but the photographing unit 102 may be provided with a focus adjustment mechanism (function) so that the photographing unit 102 can be configured to adjust the focus, for example, in response to a command from the input device 104 or automatically.

[0020] In this embodiment, shadow A scanner is used as the device, but an industrial camera (2D camera) is used instead. shadow In this case, since the camera has a wide field of view for each shooting operation, the driving unit does not translate the camera continuously, but moves the camera and the lighting unit so that the following sequence is repeated: shoot → translate the camera and lighting unit (for example, a few centimeters) → stop → shoot → translate → stop → ...

[0021] <Example of pattern configuration for pattern sheet> 3 is a diagram showing an example of the pattern configuration of the pattern sheet 1011 used in the cell observation image capturing scanner device 100 according to this embodiment. The patterns given here are merely examples, and other patterns are also applicable.

[0022] The inventors have thoroughly investigated a suitable pattern that can be applied to a scanner to obtain clear cell images, and have found that if the boundaries between light and dark are too far apart in the pattern, the cells 1102 are not captured in the middle, whereas if the boundaries between light and dark are too close, the clarity of the cells 1102 decreases. Based on this discovery, further investigations have revealed that the pattern shown in Figure 3 (pattern: pattern formed by difference in light transmittance) is applicable.

[0023] That is, for example, as shown in FIG. 3, there are a block pattern (FIG. 3A), a checker pattern (FIG. 3B), a dot pattern (FIG. 3C), a mesh pattern (FIG. 3D), a random pattern (FIG. 3E), a ripple pattern (FIG. 3F), a stripe pattern (FIG. 3G), a concentric circle pattern (FIG. 3H), a wave pattern (FIG. 3I), a hexagonal pattern with a middle fill (FIG. 3J), a dot pattern arranged in an equilateral triangle (FIG. 3K), a dot pattern arranged in an equilateral triangle with a middle fill (FIG. 3L), an equilateral triangle pattern (FIG. 3M), an equilateral triangle with a hollow (FIG. 3N), an equilateral triangle with a hollow and a middle fill (FIG. 3O), etc. The pattern does not have to be a periodic pattern. For example, it may be a so-called Penrose style pattern, or it may be a random pattern with no regularity.

[0024] For example, in the case of a block pattern (Figure 3A), the block appearance period can be 1780.9 μm (= 84px × 25400 μm / 1200 dpi), and one side of one block can be 890 μm (= 42px × 25400 μm / 1200 dpi). In addition, in the case of a stripe pattern (Figure G), the stripe appearance period can be 508.8 μm (= 24px × 25400 μm / 1200 dpi), and one stripe width can be 254.4 μm (= 12px × 25400 μm / 1200 dpi). Furthermore, in the case of a ripple pattern (Figure F), the ripple appearance period can be 508.8 μm (= 24px × 25400 μm / 1200 dpi), and one ripple width can be 254.4 μm (= 12px × 25400 μm / 1200 dpi). In addition, in the case of a wave pattern (Figure I), the wave period can be 636 μm (= 30px × 25400 μm / 1200 dpi), the amplitude can be 424 μm (= 10px × 2 × 25400 μm / 1200 dpi), and the thickness can be 318 μm (= 15px × 25400 μm / 1200 dpi).

[0025] For example, mixed patterns can also include a combination of mesh and blocks (a pattern in which a pattern like the kanji character "donburi" is repeated) or a combination of checkers, black blocks, and white blocks (checkers ± blocks) (a pattern in which a pattern like the kanji character "kai" is repeated).

[0026] By providing the pattern sheet 1011 having the above-mentioned pattern on the top (above) of the cell culture vessel 110, it becomes possible to obtain an observation image by a scanner, in which the presence and state of each individual cell 1102 can be clearly recognized. In other words, if the transparency of the bottom surface of the cell culture vessel 110 decreases, it is possible to know that the culture is progressing, but even if one can recognize this to a certain extent, it is difficult to fully utilize it for regenerative medicine. According to this embodiment, since it is possible to grasp the state of each individual cell, this technology contributes to the further development of regenerative medicine.

[0027] <Considerations on the principle of how adding a pattern sheet enables cells to be photographed> FIG. 4 is a diagram for explaining the principle by which cells can be photographed by adding a pattern sheet.

[0028] Light does not pass through the black parts of the pattern sheet 1011. Since light emitted from each point of the lighting unit 101 located above the pattern sheet 1011 is irradiated in all directions, the light passing through the transparent parts of the pattern sheet 1011 includes not only light 401 perpendicular to the pattern sheet 1011 but also light 402 obliquely.

[0029] The vertical light 401 and the diagonal light (diffused light) 402 are added together over the entire lighting unit 101, resulting in an image captured by the photographing unit 102. shadow In the image captured, the pattern of the pattern sheet 1011 appears blurred. In this case, even if vertical light 401 enters the cell 1102, it passes straight down and the cell 1102 is not visible. On the other hand, when oblique light 402 enters the cell 1102, the light is refracted due to a slight difference in the refractive index. The oblique light 402 does not reach the image it would have if the cell 1102 had not been present. shadow The different images are taken due to refraction, not due to element 403. shadow As a result, the light reaches element 404. shadow The amount of light received by element 403 is slightly reduced, and the image is taken instead. shadow Element 404 receives a small increase in the amount of light.

[0030] This change in light intensity is not that great, but shadow Element 403 and shooting shadow When the influence of the oblique light 402 is more dominant than the perpendicular light 401 at the point of the element 404, the cell 1102 is imaged at a level that can be seen. shadow The areas of the image where the light and dark patterns are blurred to create a gradation satisfy this property.

[0031] Generally, the difference in refractive index between the cell 1102 and the culture solution is small, so the refraction that occurs when the light 402 is incident on the cell 1102 in an oblique direction is small. shadow When the distance to reach the element 404 is short, the image that should arrive when there is no cell 1102 (when there is no refraction) is shadow Element 403 and shadow Since the elements 404 are the same, it becomes impossible to confirm the presence of cells in the image. shadow A sufficient distance is required between the element surfaces. Therefore, sufficient effect cannot be obtained with CIS (Contact Image Sensor) type scanners or microscopes, where this distance is very short. On the other hand, it is known that the effect can be easily obtained with CCD (Charge Coupled Device) type scanners, where the distance is sufficiently large, or with configurations such as industrial cameras connected to macro lenses.

[0032] <Comparison of observed images (experimental results)> FIG. 5 shows an image observed by microscope tiling photography (FIG. 5A), an image observed by conventional scanner photography (FIG. 5B), and an image observed by scanner photography of the present method (FIG. 5C).

[0033] The cells used in this experiment were cartilage cells, with a long side (lengthwise direction) of 50 to 80 μm and a short side (widthwise direction) of 30 to 50 μm. The size of the pattern of the pattern sheet 1011 used to photograph Figure 5C is about 3 mm or less in the direction in which the pattern is repeated and in the size of one period of the pattern.

[0034] Comparing FIG. 5A to FIG. 5C, in the observation image obtained using the microscope tiling photography (FIG. 5A), it was possible to display the cells, but as described above, the photography efficiency was poor, and the image was not smooth because multiple microscope images were stitched together. In addition, in the case of the conventional scanner photography (FIG. 5B), the photography efficiency was good and the image obtained was smooth, but it was found that the cells could not be displayed unless they were stained. Since staining the cells causes damage to the cells, the conventional scanner photography is also not preferable. In contrast, in the case of photography according to this embodiment (FIG. 5C), the photography efficiency is good and the image obtained is smooth.

[0035] The size of the pattern may be related to the size of the cells and the height at which the pattern is placed (height of the container). When a tall (deep) cell culture container (dish) 110 is used, it is not possible to capture good images of individual cells 1102 with the size of the pattern, which was good with a shallow culture container (dish) 110. shadow When the boundary of the pattern on the surface creates a simple gradation, the image can be captured well. However, when the gradations from adjacent boundaries intersect and create complex shadows, the visibility of the cells becomes poor. When visibility is poor, the pattern and the image cannot be captured. shadow As the distance between the surfaces increases, the main components of the diagonal light 402 emitted from adjacent transparent regions in the pattern intersect with each other. In this case, even if one of the intersecting light rays 402 experiences the effect of refraction due to the cells, the other light ray 402' does not experience the effect of the cells. As a result, the effect of highlighting the light and dark due to refraction is masked, reducing the visibility of the cells. Therefore, when the cell culture vessel 110 is deeper, it is necessary to increase the pattern size (the distance between the transparent regions) to avoid the main components of the diagonal light rays intersecting with each other. The main components are the components of the light ray 402 in each direction that are the closest to the image. shadow This refers to a directional component that has a non-negligible effect on the image being displayed.

[0036] <About condensation> When condensation (temperature difference with the outside) occurs inside the cell culture vessel 110, the visibility is weakened and a good observation image cannot be obtained. In this regard, applying an anti-condensation agent to the inside of the top cover of the cell culture vessel 110 will have a detrimental effect on the cells 1102.

[0037] Therefore, for example, the top cover (made of glass) of the cell culture vessel 110 may be heated by a heater. The best method for preventing condensation is to place the entire scanner device in a culture device (environment with a temperature of 37° C. and a humidity of 99%).

[0038] <Image output processing using machine learning> When an observation image is acquired using the pattern sheet 1011 as in this embodiment, a part of the pattern (blurred state) is reflected in the background of the observation image, resulting in the appearance of light and dark undulations (see FIG. 5C). Therefore, the light and dark undulations caused by the pattern can be removed by AI processing such as deep learning. For example, it is possible to convert the image into an image (without background) equivalent to an observation image obtained when the microscope tiling process is very successful. Here, an example of acquiring an observation image by such AI processing is described. Note that various types of images can be used as pair images of the captured observation image, such as an "observation image with a pattern sheet background" and an output image corresponding to this, such as an "image as obtained by a phase contrast microscope," an "image as cytoplasmic staining (for example, the cell area is green and the background is black)," or an "image as nuclear staining (only the center of the cell is blue and the rest of the cell is black)."

[0039] (i) Step 1 A culture medium 1101 and cells 1102 are placed in a cell culture vessel 110 to prepare a target.

[0040] (ii) Step 2 The target prepared in step 1 is placed in the placement position (predetermined position of the photographing unit 102) of the scanner device 100 for photographing cell observation images, and then covered with a scanner cover (illumination unit 101) with the pattern sheet 1011 attached (or printed) (or the pattern sheet 1011 alone is placed on the top lid of the cell culture vessel 110), and a cell observation image is obtained. As described above, this cell observation image has the blurred pattern of the pattern sheet 1011 reflected in the background. It goes without saying that different images are obtained depending on the type of pattern sheet (see FIG. 3).

[0041] (iii) Step 3 Prepare (obtain or create) an image that will be paired with part or all of the cell observation image obtained in step 2.

[0042] When acquiring paired images, the target in step 1 is photographed in a manner that has the desired properties. For example, paired images can be acquired by photographing with a phase-contrast microscope and performing a tiling process, or by staining the nuclei, photographing with a fluorescence microscope, and performing a tiling process to acquire paired images. In addition, when creating paired images, images with the desired properties can be created by converting the acquired images manually or by separate image processing. Note that it is necessary to prepare a sufficient number of paired images (such as multiple pairs or pairs with sufficiently large image sizes).

[0043] Fig. 6 is a diagram showing an example of an image acquired by the scanner device 100 for capturing cell observation images and an example of an image paired with the image. The paired image in Fig. 6 corresponds to a case where a paired image is created, and is an example created manually by defining the "desired property" as "a cell region extracted."

[0044] (iv) Step 4 Machine learning is performed to construct a machine learning model using the paired images prepared in step 3. Figure 7 illustrates the concept of machine learning using paired images.

[0045] For example, a "U-Net" (suitable for region extraction) is adopted as the network shape, and learning is performed using "a partial (or full) image" of the cell observation image acquired by the scanner device 100 for capturing cell observation images as input, and "an image paired with the input" as output (Deep Learning).

[0046] As another method, for example, a technique called support vector regression (SVR) can be used to perform machine learning using the input as "vectorized version of a very small subregion in a pair of input images" and the output as "the value in the pair of images that corresponds to the center of the input subregion" (traditional machine learning compared to Deep Learning).

[0047] (v) Step 5 The steps up to step 4 are the preparatory steps. Step 5 corresponds to the process during actual operation. Fig. 8 is a diagram showing the concept of inputting an actual observation image with a pattern sheet background and outputting an observation image without the background.

[0048] Using the same pattern sheet 1011 as in step 1 (pattern sheet 1011 used for machine learning), a cell observation image X captured by the scanner device 100 of a target (a target during actual operation) different from the target used for learning becomes an input for the machine learning model constructed in step 4. When such a cell observation image X is applied to the machine learning model, an image in which the "desired properties" are exhibited for the cell observation image X is obtained as an output image.

[0049] In addition, when the cell observation image X is different from the size Y assumed by the machine learning model, generally, areas of size Y are cut out one after another from inside the cell observation image X, processed sequentially, and finally integrated. It is also possible to allow overlaps during cutting out, and to perform processing such as averaging or maximum value selection during integration.

[0050] <Image analysis processing> Various image analysis processes (e.g., edge extraction process, pixel counting process to calculate area, etc.) can be performed on the cell observation image obtained by photographing or the output image obtained by machine learning to obtain various analysis results. Such analysis processes can be performed, for example, by using the computer 103 in the cell observation image photographing scanner device 100. The image analysis results can then be used as specific management indexes.

[0051] Items to be analyzed include, for example, cell position, the area of ​​each cell or the total area of ​​all cells, cell morphology, cell differentiation state, cell movement, proximity, contact, division, and death, parent-child cell lineages and cell populations of the same lineage, and proximity or contact between cell populations.

[0052] In addition, management indices calculated based on the image analysis results include, for example, cell number (current value, progress, and future prediction), proliferation rate (current value, progress, and future prediction), and differentiation rate (current value, progress, and future prediction).

[0053] For example, by obtaining an output in which the cell area is white and the background is black, and then counting the number of white areas, the number of cells can be counted. In addition, by calculating the center of gravity of each white area, the position of each cell can be grasped. Furthermore, by measuring the number of pixels in the white area, the area of ​​each cell can be grasped. Depending on the characteristics of the cells photographed, it may be possible to distinguish the cell type and the differentiation state using the shape of the area. In particular, when taking pictures in a time series by taking advantage of the characteristic of this technology that it can take pictures frequently, it is possible to grasp information such as cell movement, proximity, contact, division, and death by applying existing cell tracking technology. It is easy to calculate the proliferation rate from the change in the number of cells, and by organizing the information on division and death in a time series, it is possible to grasp the parent-child lineage of cells. In addition, by organizing the information on proximity and parent-child lineage, it is possible to grasp information such as proximity and contact about the cell population. The grasped information can be used, for example, to make a comprehensive judgment on the quality of the cell culture state, and can also be used to predict the future cell number and cell culture state based on the changes in each index over time. Such situational understanding and future predictions can be used to find a wide range of applications in this field, such as determining when to change the culture medium, reserving a date for a transplant operation based on a prediction of the date and time when the required number of cells will be obtained, and reducing costs by detecting poor culture conditions early and discontinuing the culture.

[0054] (2) Second embodiment FIG. 9 is a diagram for explaining the operation of the scanner device 200 for capturing a cell observation image according to the second embodiment.

[0055] The scanner device 200 for photographing cell observation images according to the second embodiment includes an illumination unit, a photographing unit, and a scanning unit, similar to the scanner device 100 for photographing cell observation images according to the first embodiment. shadow The scanner device 200 for cell observation and image capture according to the second embodiment includes a lighting unit, a computer, an input device, a display device, and a storage device. shadow The unit is not directly facing the object, but is placed at an angle and photographed with the lighting unit. shadowThe illumination unit is configured to move in a synchronized manner. shadow Therefore, the illumination unit irradiates parallel light (collimated light) onto the cell 1102 from above at an angle, and only the light refracted by the cell is captured. shadow The light is then incident on the unit.

[0056] By adopting such a configuration, the scanner device 200 for capturing cell observation images according to the second embodiment does not need to arrange the pattern sheets shown in Figure 3, and an image with a black background and gray to white cell areas is automatically obtained.

[0057] (3) Third embodiment FIG. 10 is a diagram for explaining the operation of the scanner device 300 for capturing a cell observation image according to the third embodiment.

[0058] The scanner device 300 for taking cell observation images according to the third embodiment has a configuration similar to that of the scanner device 200 for taking cell observation images according to the second embodiment, and takes a picture of the illumination unit. shadow The unit is not directly facing the object, but is placed at an angle and photographed with the lighting unit. shadow The illumination unit is configured to move in a synchronized manner. shadow The illumination unit is arranged so that the element is removed (or shielded). The illumination unit irradiates parallel light (collimated light) onto the cell 1102 from above at an angle, and only the light refracted by the cell is captured. shadow However, in the third embodiment, the light is incident on the imaging unit. shadow A virtual color is assigned to each unit (a photo output that outputs pixel data of any color). shadow Therefore, as shown in FIG. 10, the parallel light refracted by the cell 1102 is captured as a plurality of images with different assigned colors. shadowWhen the cells are overlapped, the light incident on each element is integrated, and an image with a color according to the amount of light incident on each element is obtained. For example, the light refracted by the cell 1102 is captured as a blue image. shadow Element and red photography shadow When the light enters the element, the colors are integrated to obtain a purple image (colorizing the image of the cell). Also, when observing a cell 1102, which has a stronger refractive index, red and green images are combined. shadow If light is incident across the element, a yellow image is obtained.

[0059] By adopting such a configuration, in the scanner device 300 for taking cell observation images according to the third embodiment, as in the second embodiment, there is no need to arrange the pattern sheets shown in Figure 3, and an image in which the background is black and the cell parts are virtually colored (the color of the image varies depending on the refractive index of the cells) can be automatically obtained.

[0060] (4) Fourth embodiment FIG. 11 is a diagram for explaining the operation of the scanner device 400 for capturing a cell observation image according to the fourth embodiment.

[0061] The scanner device 400 for photographing cell observation images according to the fourth embodiment includes an illumination unit, a photographing unit, and a scanning unit, similar to the scanner device 100 for photographing cell observation images according to the first embodiment. shadow The fourth embodiment includes a lighting unit, a computer, an input device, a display device, and a storage device. The lighting unit emits diffused light (non-collimated light) as in the first embodiment. However, the fourth embodiment differs from the first embodiment in that multiple pattern sheets (see FIG. 11B) are vertically shifted (spaced apart) and placed under the lighting unit.

[0062] The above-mentioned photograph shadow According to the principle, in areas where the light is mainly oblique, it is difficult to take images of cells. shadowIt has been found to be highly effective. As shown in FIG. 11A, pattern sheet 1 can remove most of the vertical light from the light source (lighting unit). In order to remove the vertical and near-vertical light that passes through the transparent part of pattern sheet 1, pattern sheet 2 is placed at a certain distance from pattern sheet 1. Pattern sheet 2 is basically a pattern in which the black and white of pattern sheet 1 is inverted, but the configuration of each sheet can be adjusted depending on how close to vertical light is to be removed. For example, if you want to leave only the diagonal component with a stronger angle, you can increase the shielding part of pattern sheet 2 or reduce the distance between pattern sheet 1 and pattern sheet 2. Alternatively, you can combine increasing the shielding part and shortening the distance between pattern sheets 1 and 2. As shown in FIG. 11A, it is necessary to devise the arrangement and pattern of pattern sheets 1 and 2 so that a sufficient amount of diagonal light covers the entire cell culture surface. Although two sheet pairs are shown in FIG. 11B, a set of three or more sheets may be used. In this case, three or more pattern sheets are arranged vertically at a predetermined distance apart, and the light transmitting portion of pattern sheet 1 is covered by the light blocking portions of multiple pattern sheets 2 to N.

[0063] (5) Fifth embodiment Fig. 12 is a diagram showing an example of the configuration of a pattern sheet according to the fifth embodiment. The pattern sheets shown in Figs. 12A and 12B can be used in the scanner device 100 for cell observation image capture. The pattern sheet can be configured, for example, to have a continuous change in transparency (Fig. 12A) or a stepwise change in transparency (Fig. 12B). By using these, it is possible to reduce the light irradiated onto the culture vessel (cells) from the vertical direction and relatively increase the light from the oblique direction.

[0064] The above-mentioned photograph shadow According to the principle, cells are photographed in areas where light comes mainly from oblique directions. shadowIt has been found that this method is highly effective. In the embodiments described above, the pattern sheet is composed of black (completely opaque) parts and transparent parts, but as in this embodiment, it is also possible to configure the pattern by distributing the transparency. For example, if it is desirable for the pattern not to be too prominent in the background for the convenience of subsequent analysis processing, the light and dark of the background can be suppressed and made closer to uniform by using a pattern sheet whose transparency changes continuously or in stages.

[0065] <Summary> (i) In the cell observation image capturing device (scanner and camera) according to the above-mentioned embodiment, a pattern sheet is disposed between the lighting unit and the top lid of the culture vessel. The pattern sheet can be disposed by attaching it to the glass surface of the lighting unit, or it can be printed on the glass surface of the lighting unit, or it can be formed by processing the glass surface of the lighting unit into a concave-convex shape. Furthermore, the pattern sheet may be configured as a sheet member independent of the cell observation image capturing device, and the sheet member may be placed on the top lid of the culture vessel during imaging. By configuring the imaging device in this way, it is possible to obtain a cell observation image in which each individual cell can be clearly recognized. Examples of patterns that can produce this effect include block patterns, checker patterns, dot patterns, mesh patterns, random patterns, ripple patterns, stripe patterns, wave patterns, mixed patterns which are combinations of multiple patterns, and patterns consisting of any of a number of patterns including a concentric circle pattern, a hexagonal fill pattern, an equilateral triangular dot pattern, an equilateral triangular fill dot pattern, an equilateral triangle pattern, an equilateral triangle hollow pattern, an equilateral triangle hollow fill pattern, a non-periodic pattern, a pattern with continuous changes in transparency, and a pattern with gradual changes in transparency. The imaging device used was not a scanner but a two-dimensional imager. shadowEven when using a device (such as a camera), the pattern is reflected in the background, so it is possible to align the object more accurately than before, and this can be used as a clue, thereby ensuring the continuity of the observed image. In addition, by using a camera, a much wider field of view can be captured in one shot than with a microscope, so the number of shots required is reduced, and as a result, the required time can be shortened.

[0066] (ii) In the cell observation image capturing device according to the embodiment, the focus of the capturing unit is set on the glass surface (the capturing surface) of the capturing unit, unlike a normal capturing device (scanner device). shadow The target is not aligned with the target surface (the surface on which the target is placed), but is aligned with a position shifted a predetermined distance toward the illumination unit from the culture vessel placement surface of the photographing unit (for example, above the thickness of the bottom surface of the cell culture vessel: that is, the "predetermined distance" includes at least the distance of the thickness of the culture vessel). shadow The focus of the unit is configured to match the refractive surface of the object (cell) to be photographed. This makes it possible to provide an imaging device suitable for photographing cell observation images.

[0067] (iii) The storage device of the cell observation image capturing device stores a machine learning model for outputting a converted image exhibiting desired properties from a cell observation image. At this time, a cell observation image obtained by actually photographing a culture vessel may be applied as an input to the machine learning model to obtain a converted image, which may then be displayed on the screen of an output device (display device). In this way, it becomes possible to remove light and dark undulations due to light and dark patterns reflected in the cell observation image.

[0068] In addition, a predetermined image analysis process may be executed on the cell observation image or the converted image, and a predetermined control index may be calculated and displayed on the screen of an output device (display device).

[0069] The predetermined image analysis process is, for example, a process for analyzing (iii-1) cell position, (iii-2) area of ​​each cell or total area of ​​all cells, (iii-3) cell morphology, (iii-4) differentiation state of cells, (iii-5) cell movement, proximity, contact, division, and death, (iii-6) parent-child lineage of cells and cell populations of the same lineage, or (iii-7) proximity or contact between cell populations. Also, the number of cells, proliferation rate, or differentiation rate is calculated as a predetermined control index.

[0070] (iv) In the cell observation image capturing device (scanner device) according to the second embodiment, the illumination unit captures shadow Unit photo shadow The parallel light is arranged to be obliquely irradiated onto the surface, and the parallel light travels straight ahead. shadow Unit photo shadow Only the parallel light refracted by the cells is captured. shadow Unit photo shadow This allows the light to be incident on the surface of the culture vessel, making it possible to obtain clear observation images of the cells without arranging a specific pattern between the illumination unit and the top lid of the culture vessel.

[0071] In the third embodiment, in addition to the configuration according to the second embodiment, shadow The unit is made up of multiple cameras, each with a specific color assigned to it. shadow At this time, the parallel light refracted by the cells is imaged with different colors. shadow When parallel light enters the element, it is shadow The colors of the elements are mixed to output an image.

[0072] (v) In the fourth embodiment, a predetermined pattern having different light transmittances is also arranged between the illumination unit and the top cover of the culture vessel, and the predetermined pattern is composed of a set of multiple pattern sheets arranged at a predetermined distance apart. The light-transmitting portion of the pattern sheet arranged closest to the illumination unit is covered by the light-shielding portion of the other pattern sheets. At this time, the light transmitted through the multiple pattern sheets illuminates the entire culture surface of the culture vessel. In this way, by combining multiple pattern sheets, it is possible to reduce the light from the vertical direction and relatively increase the light from an oblique direction, so that a clear cell observation image can be obtained.

[0073] (vi) Some of the functions of the present embodiment can also be realized by software program code. In this case, a storage medium on which the program code is recorded is provided to a system or device, and the computer (or CPU or MPU) of the system or device reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-mentioned embodiment, and the program code itself and the storage medium on which it is stored constitute the present disclosure. Examples of storage media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, and ROMs.

[0074] Also, an operating system (OS) running on a computer may perform all or a part of the actual processing based on the instructions of the program code, and the functions of the above-mentioned embodiments may be realized by the processing. Furthermore, after the program code read from a storage medium is written into a memory on a computer, a CPU of the computer may perform all or a part of the actual processing based on the instructions of the program code, and the functions of the above-mentioned embodiments may be realized by the processing.

[0075] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed over a network and stored in a storage means such as a hard disk or memory of the system or device, or in a storage medium such as a CD-RW or CD-R, so that when used, the computer (or CPU or MPU) of the system or device reads out and executes the program code stored in the storage means or storage medium.

[0076] It should be understood that the processes and techniques described herein are not inherently related to any particular apparatus, but may be implemented by any suitable combination of components. Furthermore, various types of general-purpose devices may be used in accordance with the disclosure described herein. It may be beneficial to build a dedicated apparatus to perform the steps of the methods described herein, and various inventions may be formed by suitable combinations of multiple components disclosed in the embodiments. For example, some components may be omitted from all components shown in the embodiments, and different components may be combined as appropriate. Although the present disclosure has been described in relation to specific examples, these are provided for technical understanding rather than for the purpose of limiting the scope of the claims in all respects. It is also believed that a person skilled in the art will readily recognize that there are numerous combinations of hardware, software, and firmware suitable for implementing the present disclosure.

[0077] Furthermore, in the above-described embodiment, the control lines and information lines are those that are considered necessary for the explanation, and not all of the control lines and information lines in the product are necessarily shown. All of the components may be connected to each other. [Explanation of symbols]

[0078] 100 Scanner device for capturing cell observation images 101 Lighting unit 102 Photography Unit 103 Computer 104 Input Devices 105 Storage device 106 Output Device 110 Cell culture vessels 1011 Pattern sheet 1101 Culture solution 1102 cells

Claims

1. A cell observation image capturing device for capturing cell observation images by placing a culture vessel carrying cells to be cultured therein, an illumination unit located above the culture vessel and configured to irradiate the culture vessel with illumination light; A photographing unit is provided, the photographing unit being located under the culture vessel and photographing the culture vessel; A predetermined pattern having a difference in light transmittance is arranged between the lighting unit and the upper cover of the culture vessel, A cell observation image capturing device in which the focus of the photographing unit is adjusted to a position shifted a predetermined distance toward the illumination unit from the culture vessel mounting surface of the photographing unit, thereby aligning the focus with the cell surface, and no tiling photography is performed.

2. In claim 1, A cell observation and image capturing device, wherein the predetermined pattern is arranged by attaching a sheet having the predetermined pattern to the light transmitting surface of the lighting unit.

3. In claim 1, A cell observation and image capture device, wherein the predetermined pattern is printed on the light transmitting surface of the illumination unit, or formed by roughening the light transmitting surface of the illumination unit.

4. In claim 1, The predetermined pattern is formed of a sheet member independent of the cell observation image capturing device, and the sheet member is placed on the top lid of the culture vessel when capturing images of the cell observation image capturing device.

5. In any one of claims 1 to 4, A cell observation image capturing device, wherein the specified pattern is any one of a plurality of patterns including a block pattern, a checkered pattern, a dot pattern, a mesh pattern, a random pattern, a ripple pattern, a stripe pattern, a wave pattern, a concentric circle pattern, a filled hexagonal pattern, a dot pattern arranged in equilateral triangles, a filled dot pattern arranged in equilateral triangles, a regular triangle pattern, a hollowed out regular triangle pattern, a hollowed out regular triangle pattern, a pattern without periodicity, a pattern having a continuous change in transparency, and a pattern having a gradual change in transparency.

6. In any one of claims 1 to 5, The present invention further includes a computer that executes various calculations, a storage unit that stores the results of the calculations performed by the computer, and an output unit that outputs the results of the calculations, The storage unit stores a machine learning model for outputting a converted image exhibiting desired properties from the cell observation image, The computer applies the cell observation image obtained by actually photographing the culture vessel as an input to the machine learning model to obtain the transformed image, and outputs the transformed image to the output section, in a cell observation image capturing device.

7. In claim 6, The computer executes a predetermined image analysis process on the cell observation image, calculates a predetermined control index, and outputs it to the output unit.

8. In claim 6, The computer executes a predetermined image analysis process on the converted image, calculates a predetermined control index, and outputs it to the output unit.

9. In claim 7 or 8, The computer uses the specified image analysis processing to analyze (i) cell position, (ii) area of ​​each cell or the total area of ​​all cells, (iii) cell morphology, (iv) differentiation state of cells, (v) cell movement, proximity, contact, division, and death, (vi) parent-child cell lineage and cell populations of the same lineage, or (vii) proximity or contact between cell populations.

10. In any one of claims 7 to 9, The computer calculates the number of cells, the degree of proliferation, or the differentiation rate as the predetermined control index.

11. A cell observation image capturing device for capturing cell observation images by placing a culture vessel carrying cells to be cultured therein, an illumination unit located above the culture vessel and configured to irradiate the culture vessel with illumination light; A photographing unit is provided, the photographing unit being located under the culture vessel and photographing the culture vessel; A predetermined pattern having a difference in light transmittance is arranged between the lighting unit and the upper cover of the culture vessel, The predetermined pattern is composed of a plurality of pattern sheet sets arranged at a predetermined distance apart, and a light-transmitting portion of the pattern sheet arranged closest to the lighting unit is covered by a light-shielding portion of another pattern sheet, A cell observation image capturing device in which the focus of the photographing unit is adjusted to a position shifted a predetermined distance toward the illumination unit from the culture vessel mounting surface of the photographing unit, thereby aligning the focus with the cell surface, and no tiling photography is performed.

12. In claim 11, The set of multiple pattern sheets is arranged so that light transmitted through the multiple pattern sheets illuminates the entire culture surface of the culture vessel, in this cell observation screen imaging device.

13. A cell observation image acquisition method for acquiring an observation image of a cell supported in a culture vessel using a cell observation image acquisition device, comprising: Illuminating the culture vessel with illumination light by an illumination unit located above the culture vessel in the cell observation image capture device through a predetermined pattern disposed between the illumination unit and an upper cover of the culture vessel, the pattern having a difference in light transmittance; and capturing an image of the culture vessel by an imaging unit located below the culture vessel in the cell observation image capturing device, A cell observation image acquisition method in which the focus of the photographing unit is adjusted to a position shifted a predetermined distance toward the illumination unit from the culture vessel mounting surface of the photographing unit, thereby aligning the focus with the cell surface, and no tiling photography is performed.

14. In claim 13, A cell observation image acquisition method, wherein the predetermined pattern is arranged by attaching a sheet having the predetermined pattern to the light-transmitting surface of the lighting unit.

15. In claim 13, A cell observation image acquiring method, wherein the predetermined pattern is printed on the light transmitting surface of the lighting unit, or formed by roughening the light transmitting surface of the lighting unit.

16. In claim 13, A cell observation image acquiring method, in which the predetermined pattern is composed of a sheet member independent of the cell observation image capturing device, and the sheet member is placed on the top lid of the culture vessel during capturing.

17. In any one of claims 13 to 16, A cell observation image acquisition method, wherein the specified pattern is any one of a plurality of patterns including a block pattern, a checkered pattern, a dot pattern, a mesh pattern, a random pattern, a ripple pattern, a stripe pattern, a wave pattern, a concentric circle pattern, a filled hexagonal pattern, a dot pattern arranged in equilateral triangles, a filled dot pattern arranged in equilateral triangles, a regular triangle pattern, a hollowed out regular triangle pattern, a hollowed out regular triangle pattern, a pattern with no periodicity, a pattern having a continuous change in transparency, and a pattern having a gradual change in transparency.

18. In any one of claims 13 to 17, further comprising: A computer reads in a machine learning model stored in a storage unit, the machine learning model being for outputting a converted image exhibiting a desired property from an observation image of the cell; the computer applies an observation image of the cell obtained by actually photographing the culture vessel as an input to the machine learning model to obtain the converted image, and outputs the converted image to an output unit; A method for acquiring a cell observation image, comprising:

19. In claim 18, further comprising: The cell observation image acquisition method includes the computer executing a predetermined image analysis process on the cell observation image, calculating a predetermined management index, and outputting the calculated management index to the output unit.

20. In claim 18, further comprising: The cell observation image acquisition method includes the computer executing a predetermined image analysis process on the converted image, calculating a predetermined control index, and outputting the calculated control index to the output unit.

21. In claim 19 or 20, The computer uses the specified image analysis process to analyze (i) cell position, (ii) the area of ​​each cell or the total area of ​​all cells, (iii) cell morphology, (iv) cell differentiation state, (v) cell movement, proximity, contact, division, and death, (vi) parent-child cell lineage and cell populations of the same lineage, or (vii) proximity or contact between cell populations.

22. In any one of claims 19 to 21, The computer calculates the number of cells, the degree of proliferation, or the rate of differentiation as the predetermined control index.

23. A cell observation image acquisition method for acquiring an observation image of a cell supported in a culture vessel using a cell observation image acquisition device, comprising: Illuminating the culture vessel with illumination light by an illumination unit located above the culture vessel in the cell observation image capture device through a predetermined pattern disposed between the illumination unit and an upper cover of the culture vessel, the pattern having a difference in light transmittance; and capturing an image of the culture vessel illuminated by the illumination light by an imaging unit located under the culture vessel in the cell observation image capturing device, The predetermined pattern is composed of a plurality of pattern sheet sets arranged at a predetermined distance apart, and a light-transmitting portion of the pattern sheet arranged closest to the lighting unit is covered by a light-shielding portion of another pattern sheet, A cell observation image acquisition method in which the focus of the photographing unit is adjusted to a position shifted a predetermined distance toward the illumination unit from the culture vessel mounting surface of the photographing unit, thereby aligning the focus with the cell surface, and no tiling photography is performed.

24. In claim 23, A cell observation screen acquiring method, wherein the plurality of pattern sheet sets are arranged so that light transmitted through the plurality of pattern sheets illuminates the entire culture surface of the culture vessel.

Images