Inspection method, computer-readable recording medium

Abstract

Provided are an inspection method and a computer-readable recording medium for assisting in the reliability evaluation of the counting function of an imaging system that captures a phase object. In an inspection method for inspecting the counting function of an imaging system (1) that captures a phase object, under the condition of determining the number of counting target regions (212) located within the field of view of the imaging system (1), a standard plate (210) including a base region (211) and a plurality of counting target regions (212) having a phase amount different from that of the base region (211) in the thickness direction is captured, the counting target regions (212) included in the image of the captured standard plate (210) are counted, and at least the counting result of the counting target regions (212) is output.

Description

Technical Field

[0001] The disclosure of this specification relates to an inspection method, a computer-readable recording medium, and a standard plate. Background Art

[0002] As a method for counting cells, there are known methods such as a method of counting cells using a cell counting plate, a method of counting cells arranged in a line using a flow cytometer, and a method of counting cells based on an image by analyzing an image of cells. The method of counting cells based on an image can be performed even during cultivation, and thus is suitable for uses such as counting cells during cultivation to monitor the cultivation state.

[0003] Regarding techniques for accurately counting cells based on an image, various proposals have been made. For example, Patent Document 1 describes a technique for suppressing a counting error caused by misidentifying noise included in an image as a cell nucleus by adjusting a threshold value in the frequency domain of a band-pass filter.

[0004] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2017-097227

[0005] The technique described in Patent Document 1 is a technique for accurately counting cells by suppressing the influence of noise included in an image, and can be understood as a technique for improving the quality of an image afterwards for easy analysis. Thus, the quality of an image is important for appropriately counting an object, but in the case of photographing a phase object such as a cell, for example, the quality of the image varies greatly depending on the illumination conditions. When the illumination conditions are inappropriate, a low-quality image is obtained. Therefore, even if the technique described in Patent Document 1 is applied to suppress the influence of noise included in the image, sufficient image quality cannot be expected. Accordingly, there is a need for a technique for ensuring the reliability of a counting system by preliminarily confirming that an appropriate image can be obtained. Summary of the Invention

[0006] In view of the above actual situation, an object of one aspect of the present invention is to provide a technique for assisting in evaluating the reliability of a counting function of an imaging system that photographs a phase object.

[0007] An inspection method according to one mode of the present invention inspects a counting function of an imaging system that photographs a phase object, and is characterized in that: a standard plate including a base region and a plurality of counting target regions is photographed under a condition of determining the number of counting target regions located within a field of view of the imaging system among the plurality of counting target regions, the phase amount in the thickness direction of the counting target regions being different from that of the base region; the counting target regions included in the image of the photographed standard plate are counted; and at least the counting result of the counting target regions is output.

[0008] A computer-readable recording medium according to one embodiment of the present invention stores a program for checking the counting function of an imaging system for imaging a phase object. The program causes a computer of the imaging system to perform the following processes: causing an imaging device of the imaging system to image a standard plate including a base region and a plurality of counting target regions under conditions for determining the number of counting target regions within the field of view of the imaging system, the phase amount in the thickness direction of the counting target regions being different from that of the base region; counting the counting target regions included in the image of the captured standard plate; and outputting at least the counting result of the counting target regions.

[0009] A standard plate according to one embodiment of the present invention is used for checking the counting function of an imaging system for imaging a phase object. The standard plate includes: a base region formed of a phase object; and a plurality of counting target regions formed of phase objects, the phase amount in the thickness direction of the plurality of counting target regions being different from that of the base region, and the plurality of counting target regions being regularly arranged within the standard plate.

[0010] According to the above embodiment, a technique for assisting in evaluating the reliability of the counting function of an imaging system for imaging a phase object can be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 FIG. illustrates the configuration of the imaging system.

[0012] Figure 2 FIG. illustrates the configuration of the imaging device.

[0013] Figure 3 FIG. illustrates the configuration of the control device.

[0014] Figure 4 FIG. is an example of a flowchart of an inspection method performed by the imaging system.

[0015] Figure 5 FIG. is a diagram for explaining a jig used in the inspection of the imaging system.

[0016] Figure 6 FIG. illustrates the arrangement of the standard plate in the inspection of the imaging system.

[0017] Figure 7 FIG. is a perspective view showing an example of a standard plate for counting function inspection.

[0018] Figure 8 FIG. is Figure 7 a schematic cross-sectional view of the standard plate shown.

[0019] Figure 9 FIG. isFigure 7 A top schematic view of the standard plate shown.

[0020] Figure 10 It is a top schematic view showing an example of a standard plate for checking the density meter measurement function.

[0021] Figure 11 It is a figure showing the output information of the control device.

[0022] Figure 12 It is a cross-sectional schematic view showing another example of a standard plate for checking the counting function.

[0023] Figure 13 It is a cross-sectional schematic view showing yet another example of a standard plate for checking the counting function.

[0024] Figure 14 It is a top schematic view showing another example of a standard plate for checking the counting function.

[0025] Figure 15 It is a top schematic view showing yet another example of a standard plate for checking the counting function.

[0026] Figure 16 It is a top schematic view showing yet another example of a standard plate for checking the counting function.

[0027] Figure 17 It is a top schematic view showing yet another example of a standard plate for checking the counting function.

[0028] Figure 18 It is a figure showing the structure of the microscope system.

[0029] Reference numeral description

[0030] 1: Imaging system; 2: Microscope system; 10: Imaging device; 12: Stage; 13: Imaging unit; 14: Light source; 20: Incubator; 30: Control device; 31: Processor; 32: Memory; 33: Auxiliary storage device; 60: Positioning jig; 61: Transmission part; 62: Support part; 63: Light-shielding part; 70: Light-shielding frame; 100, 200, 300: Container; 211: Substrate area; 212, 212a, 212b, 213, 214: Counting target area; 212c: Area outside the counting target; 210, 220, 230, 240, 250, 260, 270, 280, 290, 295, 310: Standard plate; 311: First area; 312: Second area; F: Field of view; M: Positioning mark. Detailed implementation

[0031] The imaging system described in this specification is a system for photographing phase objects, and is a system mainly used in biological fields such as processing cells. This imaging system at least has a function of counting the phase objects of the objects included in the image. Therefore, this imaging system can be used, for example, to obtain an image of cultured cells cultured in a controlled environment in an incubator and to grasp the quality of the culture state based on the number of cultured cells (cell count) counted from this image. In addition, the imaging system may also have a function of measuring the area ratio (hereinafter referred to as density) occupied by the phase object in the image. In addition, the phase objects to be counted are not limited to cells, and cell aggregates can also be counted. For example, bacteria (more precisely, colonies formed by bacterial cells) can be counted, or cell colonies formed by cells other than bacterial cells can be counted.

[0032] Figure 1 FIG. illustrates the structure of the imaging system. Figure 2 FIG. illustrates the structure of the imaging device. Figure 3 FIG. illustrates the structure of the control device. Hereinafter, with reference to Figures 1 to 3 FIG., the structure of the imaging system 1 shown in Figure 1 will be described.

[0033] As shown in Figure 1 FIG., the imaging system 1 includes an imaging device 10 and a control device 30 placed in an incubator 20. In the imaging system 1, the control device 30 counts cells based on the image obtained by the imaging device 10, and uses the obtained cell count to assist the user in monitoring cell culture. In addition, the control device 30 can confirm whether the counting function of the imaging system 1 is working properly by executing an inspection program of the imaging system 1 before the start of culture or the like and outputting the obtained inspection result, thereby assisting in ensuring the reliability of the counting function. And the control device 30 communicates with the imaging device 10 and the client terminals (client terminal 40, client terminal 50). In addition, the imaging system 1 may also include the incubator 20 and the client terminals.

[0034] As shown in Figure 1 FIG., a culture container 100 is placed on the imaging device 10 arranged in the incubator 20. The culture container 100 is not particularly limited, and is, for example, a petri dish, a flask, a microplate, or the like. As shown in Figure 2 FIG., a culture medium CM and cells C as cultured cells are accommodated in the culture container 100. In addition, without particular limitation, the culture medium CM is, for example, a solution containing calf serum, and the cells C are, for example, mesenchymal stem cells or iPS cells.

[0035] The imaging device 10 captures an image of the cells C accommodated in the culture vessel 100 (hereinafter, referred to as a cell image). The imaging device 10 transmits the captured image to the control device 30. The communication between the imaging device 10 and the control device 30 can be either wired communication or wireless communication. The observation method used by the imaging device 10 to capture the image may be any method suitable for observing phase objects such as cells. For example, it may be any of phase difference observation method, differential interference observation method, oblique illumination observation method, bright field observation, and dark field observation method.

[0036] More specifically, as Figure 2 shown, the imaging device 10 includes a housing 11 and a stage 12 on which the culture vessel 100 is placed. The imaging device 10 also has an imaging unit 13 below the stage 12 and inside the housing 11, a scanning mechanism 16 that moves the imaging unit 13, and a control substrate (not shown) that controls the imaging unit 13 and the scanning mechanism 16. The imaging unit 13 is provided with an imaging element 14, a light source 15, and an optical system (not shown), etc.

[0037] The imaging element 14 is, for example, a CCD (Charge-Coupled Device) image sensor, a CMOS (Complementary MOS) image sensor, or the like. The light source 15 is, for example, a light-emitting diode (LED) or the like, and illuminates the culture vessel 100 from below the stage 12. The light source 15 may also be disposed opposite to the imaging element 14 with an intervening space. In addition, the light source 15 may emit white light. For example, the light source 15 may also selectively emit light of any wavelength among R (red), G (green), and B (blue) by switching the light of the wavelengths of these three colors. In the imaging device 10, the light emitted from the light source 15 passes through the bottom surface of the culture vessel 100, and a part of the light reflected from the upper surface of the culture vessel 100 passes through the cells C in the culture vessel 100. The optical system uses the light passing through the cells C in the culture vessel 100 to form an optical image of the cells C on the imaging element 14.

[0038] The scanning mechanism 16 is controlled by a control substrate, whereby the imaging unit 13 is moved in a direction (XY direction) perpendicular to the optical axis of the optical system. The scanning mechanism 16 moves the imaging unit 13 in the XY direction, whereby the imaging device 10 can change the imaging range (i.e., the field of view of the imaging system 1). The scanning mechanism 16 can move the imaging unit 13 in the optical axis direction (Z direction) of the optical system, and the imaging device 10 can use the scanning mechanism 16 to drive the entire optical system to adjust the focus position. The scanning mechanism 16 is, for example, a motor within the imaging unit 13. In addition, the imaging device 10 can also adjust the focus position by moving at least one lens included in the optical system in the optical axis direction. Further, instead of moving the lens, a variable focus lens capable of changing the lens shape can be used to adjust the focus position. The control substrate may include a specially designed circuit, for example, an ASIC (Application Specific Integrated Circuit; integrated circuit for specific purposes), etc. Alternatively, an FPGA (Field-Programmable Gate Array) can be used to constitute it.

[0039] The control device 30 is a computer that controls the imaging system 1. As Figure 3 shown, the control device 30 includes a processor 31, a memory 32, an auxiliary storage device 33, an input device 34, an output device 35, a removable recording medium drive device 36 for driving a removable recording medium 39, a communication module 37, and a bus 38. The auxiliary storage device 33 and the removable recording medium 39 are each an example of a non-transitory computer-readable recording medium on which a program is recorded.

[0040] The processor 31 is, for example, a circuitry including a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc. The processor 31 expands the program stored in the auxiliary storage device 33 or the removable recording medium 39 into the memory 32 and then executes it, thereby performing the pre-programmed processing.

[0041] The memory 32 is, for example, an arbitrary semiconductor memory such as a RAM (Random Access Memory). When the program is executed, the memory 32 functions as a working memory for storing the program or data stored in the auxiliary storage device 33 or the removable recording medium 39. The auxiliary storage device 33 is, for example, a non-volatile memory such as a hard disk or a flash memory. The auxiliary storage device 33 is mainly used for storing various data and programs.

[0042] The removable recording medium drive device 36 houses the removable recording medium 39. The removable recording medium drive device 36 can output the data stored in the memory 32 or the auxiliary storage device 33 to the removable recording medium 39, and can read programs and data from the removable recording medium 39. The removable recording medium 39 is any portable recording medium. The removable recording medium 39 includes, for example, an SD card, a USB (Universal Serial Bus) flash memory, a CD (Compact Disc), a DVD (Digital Versatile Disc), and the like.

[0043] The input device 34 is a keyboard, a mouse, or the like. The output device 35 is a display device, a printer, or the like. The communication module 37 is, for example, a wired communication module that communicates with the imaging device 10 connected via an external port. Alternatively, the communication module 37 can be a wireless communication module. The bus 38 connects the processor 31, the memory 32, the auxiliary storage device 33, etc. so that they can mutually transmit and receive data.

[0044] Figure 3 The structure shown is an example of the hardware structure of the control device 30. The control device 30 is not limited to this structure. The control device 30 can be a general-purpose device or a dedicated device. The control device 30 can include, for example, a specially designed circuit, such as an ASIC (Application Specific Integrated Circuit). In addition, the control device 30 can also be configured using an FPGA (Field-Programmable Gate Array).

[0045] The client terminal 40 is, for example, a notebook computer. The client terminal 50 is, for example, a tablet computer. Additionally, the client terminal can also serve as the input device 34 and the output device 35, and the input device 34 and the output device 35 may not be included in the control device 30. The control device 30 can input information according to requests from the client terminals (client terminal 40, client terminal 50), and the control device 30 can also output information to the client terminals according to requests from the client terminals. The client terminal only needs to have a display unit for displaying the information received from the control device 30, and can be, for example, a desktop computer, a smart phone, or the like.

[0046] In the imaging system 1 configured as described above, for example, during the culturing of cells C, the control device 30 sends an imaging instruction to the imaging device 10. The imaging device 10 captures an image of the cells C according to the instruction from the control device 30 and sends the obtained image to the control device 30. The control device 30 receives the image from the imaging device 10, counts the cells C included in the image, and calculates the number of cells. Also, the area ratio of the cells C in the image, i.e., the cell density, can be measured based on the received image.

[0047] In addition, the counting process based on the image can also be performed using a learned model trained by deep learning or the like. In this case, the control device 30 can also distinguish and use multiple learned models according to the characteristics of the objects to be counted. Specifically, for example, the control device 30 can also switch the learned model according to the size or type of the cells to be counted. Similarly, the density measurement process based on the image can also be performed using a learned model trained by deep learning or the like.

[0048] The control device 30 notifies the user of the culture status according to the number of cells. Specifically, for example, the control device 30 can cause the output device 35 to display the trend of the number of cells, or can cause the output device 35 to display the quality of the culture status determined based on the number of cells or the trend of the number of cells. In addition, the control device 30 can also send information to the user's terminal via email or the like instead of displaying the information. In addition, the control device 30 can also notify the user of the culture status based on the number of cells and the cell density. As a notification method, for example, push notification or the like can also be used. Thus, the user can grasp the abnormality of the culture status earlier. In addition, even if the user is located far from the culture site, the user can grasp the culture status.

[0049] And, an inspection program for ensuring the reliability of the counting function of the imaging system 1 is installed in the imaging system 1. The inspection program can be executed by the control device 30, for example, before the imaging system 1 is shipped from the factory, and can also be executed by the control device 30 at the initial import of the imaging system 1 at the destination of shipment. In addition, the inspection program can also be executed by the control device 30 at an arbitrary timing according to the user's instruction.

[0050] Figure 4 is an example of a flowchart of an inspection method performed by the imaging system. Figure 5 is a diagram for explaining a jig used in the inspection of the imaging system. Figure 6 is a diagram illustrating the arrangement of a standard plate in the inspection of the imaging system. Figure 7 is a perspective view showing an example of a standard plate for the counting function inspection. Figure 8 is Figure 7 a schematic cross-sectional view of the standard plate shown. Figure 9 isFigure 7 A top schematic view of the standard plate shown. Figure 10 It is a top schematic view showing an example of a standard plate for checking the density meter measurement function. Figure 11 It is a diagram showing the output information of the control device. Hereinafter, with reference to Figures 4 to 11 , a method for checking the counting function of the imaging system 1 by executing a check program will be described.

[0051] The check performed by the imaging system 1 is carried out by using a standard plate that contains counting objects in a known quantity and density in advance. The size of the field of view of the imaging system 1 can be determined according to the imaging magnification of the imaging system 1. Therefore, if the specifications of the standard plate, such as the quantity or density of the counting objects contained in the standard plate, are known, the quantity of the counting objects contained in the image obtained by photographing the standard plate can also be determined. The imaging system 1 utilizes this point to compare the quantity of the counting objects counted through image analysis processing with the quantity of the counting objects determined according to the specifications of the standard plate and the settings of the imaging system 1, thereby checking the reliability of the counting function of the imaging system 1. In addition, the imaging system 1 can also compare the density of the counting objects measured through image analysis processing with the density of the counting objects determined according to the specifications of the standard plate, thereby checking the reliability of the density measurement function of the imaging system 1.

[0052] More specifically, in the check, as Figure 4 shown, the imaging system 1 performs the process of photographing the standard plate arranged in the container (step S1), the process of counting the counting objects contained in the standard plate (step S2), and the process of outputting the obtained counting result (step S3). Hereinafter, each process from step S1 to step S3 will be described in more detail.

[0053] In step S1, first, as Figure 5 shown, the inspector arranges the positioning jig 60 at a specified position on the stage 12 of the imaging device 10 and fixes it to the imaging device 10, for example, by screws or positioning pins. As Figure 6 shown, the positioning jig 60 includes a light-shielding portion 63 and a transmissive portion 61, and the transmissive portion 61 has a shape recessed with respect to the light-shielding portion 63. The imaging device 10 may also have protrusions or mounting holes for pre-positioning the jig.

[0054] When the positioning jig 60 is fixed, the inspector arranges two containers (container 200, container 300) in the transmission part 61. The container 200 is a container for checking the counting function, and four standard plates (standard plate 210, standard plate 220, standard plate 230, standard plate 240) are placed on the bottom surface inside the container 200. The container 300 is a container for checking the densitometer measurement function, and the standard plate 310 is placed on the bottom surface inside the container 300. Also, inside the container 200 and the container 300, in order to approximate the environment inside the container in cell culture, a liquid such as water is covered from above the standard plate. In addition, as the container 200 and the container 300, for example, a petri dish with a diameter of 50 mm or the like, a container having a predetermined size, is used. Thus, the container 200 and the container 300 are supported by the support part 62 formed on the transmission part 61, and as a result, they are fixed at a prescribed position relative to the positioning jig 60, and further fixed at a prescribed position relative to the imaging device 10. Alternatively, a jig having a container provided thereon may be prepared in advance.

[0055] The standard plate 210 is a standard plate for checking the counting function of the imaging system 1 that photographs phase objects, as Figure 7 shown, and has a structure in which a plurality of microlenses are formed on a transparent flat plate. More specifically, as Figure 8 shown, the standard plate 210 includes a base area 211 that is a part exposed on the flat plate surface and a plurality of counting target areas 212 that are parts where microlenses are formed on the flat plate. That is, each of the plurality of counting target areas 212 includes a microlens.

[0056] Both the base area 211 and the counting target areas 212 are made of an optical material such as glass or transparent plastic, for example. That is, they are made of a transparent phase object. Also, the thickness D1 of the base area 211 is different from the thickness D2 of the counting target areas 212. Therefore, in the base area 211 and the counting target areas 212, the phase amount in the thickness direction is different. Thus, by photographing the standard plate 210 using the phase difference observation method and the differential interference observation method that visualize the phase difference, an image capable of identifying the base area 211 and the counting target areas 212 can be obtained. In addition, in the bright field observation method, the contrast is also emphasized by reducing the aperture diaphragm, and an image capable of identifying the base area 211 and the counting target areas 212 can be obtained. Also, in the dark field observation method and the oblique illumination observation method, since the microlenses have a three-dimensional shape protruding from the flat plate, an image capable of identifying the base area 211 and the counting target areas 212 can be obtained.

[0057] And, as Figure 9As shown, a plurality of microlenses are formed on a flat plate at a certain fixed interval. By regularly arranging a plurality of counting object regions 212 within the standard plate 210, the number and density of the counting object regions 212 serving as counting objects are known. Therefore, if the imaging magnification of the imaging system 1 is known, regardless of the position of the field of view of the imaging system 1 on the standard plate 210, the number of counting object regions 212 located within the field of view can be determined. Thus, the number of counting object regions 212 included in the image can be determined with sufficient precision and accuracy without analyzing the image. Precision refers to the scale of the deviation between measurements, and accuracy refers to the scale indicating that the measured value is a value close to the true value. Additionally, strictly speaking, depending on the position of the field of view, the number of counting object regions 212 may vary slightly, but the magnitude of this variation is known. Therefore, by setting the known magnitude of the variation as the allowable error, the reliability of the counting function can be checked.

[0058] Similar to the standard plate 210, the standard plates 220 to 240 are standard plates for checking the counting function of the imaging system 1 for photographing phase objects. The standard plates 220 to 240 differ from the standard plate 210 in that they include counting object regions of different sizes from the counting object regions 212. In other respects, specifically, for example, the fact that a plurality of microlenses are formed on the flat plate, the number and density of the counting object regions are known, and the number of counting object regions located within the field of view can be determined, etc. are the same as those of the standard plate 210.

[0059] The standard plate 310 is a standard plate for checking the density measurement function of the imaging system 1 for photographing phase objects, as Figure 10 shown, and includes two regions (the first region 311, the second region 312) with different surface characteristics. Both the first region 311 and the second region 312 are made of an optical material such as glass or transparent plastic, for example. That is, they are both composed of transparent phase objects.

[0060] The first region 311 is a region outside the counting object and corresponds to the base region 211 of the standard plate 210. In contrast, the second region 312 is a region of the counting object and corresponds to the counting object region 212 of the standard plate 210. Specifically, the first region 311 is, for example, a part with a relatively high flatness on the flat plate surface. The second region 312 is a part that forms irregularities through, for example, sandblasting and functions as a diffuser plate as a result.

[0061] By making the standard plate 310 have a second region 312 of a known size, if the photographing magnification of the imaging system 1 is known, the area ratio of the field of view F to the second region 312 is also known. Moreover, as long as the imaging process is performed under the condition that the entire second region 312 converges on the field of view F, regardless of the position of the field of view of the imaging system 1 on the standard plate 310, the area ratio can be uniquely determined based on the photographing magnification. Therefore, the area ratio (density) of the second region 312 included in the image can be determined with sufficient accuracy and precision without analyzing the image.

[0062] When the containers 200 and 300 are fixed to the positioning jig 60, as Figure 5 shown, the inspector arranges the light-shielding frame 70 on the positioning jig 60 and covers the transmissive portion 61 with the light-shielding frame 70. Thereby, it is possible to prevent light from outside the imaging device 10 from entering below the stage 12, and thus to ensure that the standard plate can be photographed under certain conditions without being affected by the external environment. Therefore, it is possible to accurately evaluate the illumination conditions using the light source 15 inside the imaging device 10. In addition, in the case of an observation using the oblique observation method, the illumination light can also be reflected by the light-shielding frame 70.

[0063] When the above preparations are completed, in step S1, the imaging device 10 photographs the standard plate according to the imaging instruction from the control device 30 using any one of the phase difference observation method, differential interference observation method, oblique observation method, bright field observation method, and dark field observation method.

[0064] Specifically, the imaging device 10 photographs the standard plates 210 to 240 under the condition of determining the number of counting target regions located within the field of view of the imaging system 1. This condition includes the case where a plurality of counting target regions are regularly arranged within the standard plate and the case where the photographing magnification of the imaging system is known. Moreover, the imaging device 10 photographs the standard plate 310 under the condition of determining the area ratio of the counting target regions located within the field of view of the imaging system 1. This condition includes the case where the standard plate 310 converges on the field of view and the case where the photographing magnification of the imaging system is known. Then, the imaging device 10 transmits the acquired image to the control device 30.

[0065] More specifically, the control device 30 first moves the imaging unit 13 successively below the four standard plates inside the container 200. After the movement, the control device 30 causes the imaging device 10 to perform autofocus processing and imaging processing at each position. By repeating these processes a plurality of times (e.g., 10 times) without changing the imaging magnification, the imaging device 10 acquires a plurality of images of the standard plate 210, the standard plate 220, the standard plate 230, and the standard plate 240, and transmits the acquired images to the control device 30. Then, the control device 30 moves the imaging unit 13 below the standard plate 310 inside the container 300. The control device 30 performs autofocus processing and imaging processing at the moved position. At this time, by repeating the imaging process a plurality of times (e.g., 10 times) without changing the imaging magnification, a plurality of images of the standard plate 310 are acquired and transmitted to the control device 30 as a plurality of images.

[0066] When step S1 ends, the control device 30 analyzes the images received from the imaging device 10 and performs a counting process (step S2). Additionally, the control device 30 may also perform density measurement processing together with the counting process in step S2.

[0067] Specifically, the control device 30 first uses the learned model to analyze each of the plurality of images of the standard plate 210 and counts the counting target regions 212 included in each image. Additionally, the learned model is an example of an algorithm for counting the counting target regions included in an image. The control device 30 may also use an image processing algorithm other than machine learning to count the counting target regions 212. Then, the control device 30 calculates the average (Mean), standard deviation (Std deviation), coefficient of variation (CV), error (Error Bound) of the average with respect to the known number of counting target regions, etc. of the plurality of numbers obtained through the multiple counting processes. Here, the coefficient of variation is a value obtained by dividing the standard deviation by the average.

[0068] Then, the control device 30 switches the learned model used in the analysis according to the sizes of the plurality of counting target regions included in each standard plate and performs the same analysis on each of the plurality of images of the other standard plates (from the standard plate 220 to the standard plate 240). That is, the counting target regions are counted to obtain a counting result (average), and further, an evaluation result (standard deviation, coefficient of variation, error) is calculated.

[0069] The control device 30 switches the learned model used in the analysis from the model for counting the counted object regions to the model for measuring the area of the counted object regions. Then, the control device 30 uses the learned model for measuring the area of the counted object regions to analyze each of the multiple images of the standard plate 310, and measures the area ratio of the second region 312 in the image. And, the control device 30 calculates the average value (Mean), standard deviation (Std deviation), coefficient of variation (CV), error (ErrorBound) of the average value with respect to the known area ratio, etc. of the obtained multiple area ratios.

[0070] When step S2 ends, finally, the control device 30 outputs the counting result as the result of the counting process in step S2 (step S3) and ends. Figure 4 The inspection process shown. In addition, in step S3, the control device 30 can output, in addition to the counting result, at least the evaluation result of the counting function evaluated based on the counting result. Further, the control device 30 can output the density measurement result together with the counting result, and can also output at least the evaluation result of the density measurement function evaluated based on the density counting result. That is, in step S3, the control device 30 only needs to output at least the counting result.

[0071] Specifically, the control device 30 outputs the log file containing the Figure 11 information shown in Table T1 to a specified area of the auxiliary storage device 33. In addition, the control device 30 can further output the log file containing the Figure 11 information shown in Table T2 to a specified area of the auxiliary storage device 33.

[0072] In the information shown in Table T1, for each of the standard plates 210 to 240, it includes the average value (Mean), standard deviation (Std deviation), coefficient of variation (CV), error (Error Bound) of the average value with respect to the known number of counted object regions, OK / NG. Among them, the average value is the counting result itself. The standard deviation, coefficient of variation, error, and OK / NG as information other than the average value are the evaluation results of the counting function evaluated at least based on the counting result. And when classified in detail, the error in the evaluation result of the counting function is the evaluation result evaluated based on the counting result and the known number of counted object regions, and is the first evaluation result related to the accuracy of counting indicating how close the counting result is to the true value. On the other hand, the standard deviation and the coefficient of variation are the evaluation results evaluated based on the counting result, and are the second evaluation results related to the precision of counting indicating the deviation of the counting result. Thus, the evaluation result of the counting function includes the first evaluation result and the second evaluation result.

[0073] In addition, OK / NG is a comprehensive evaluation related to the reliability of the measurement function of the imaging system 1. "OK" means that, for example, the coefficient of variation is within a specified range (e.g., within 5%) and the error is within a specified range (e.g., within 20%), and "NG" means that, for example, the above conditions are not met. In addition, the conditions for determining "OK" and "NG" are not particularly limited to the above examples and can also be programmed to be adjustable.

[0074] In the information shown in Table T2, for the standard plate 310, it includes the mean, standard deviation, coefficient of variation (CV), error bound of the mean with respect to the known number of counting target areas, and OK / NG of the area ratio of the counting target area. Regarding each item, it is the same as the information shown in Table T1.

[0075] After Figure 4 the inspection process shown is completed, the inspector can confirm the reliability of the measurement function and density measurement function of the imaging system 1 by checking the information written in the log file. When the inspector determines that the reliability of each function is insufficient based on the information written in the log file, the inspector can also adjust the lighting conditions and perform the inspection again.

[0076] For example, if Figure 11 the results shown are obtained, it can be confirmed that the imaging system 1 has exhibited a reliable level of measurement performance for the standard plates 210, 220, and 240. Therefore, when counting phase objects of the size corresponding to these standard plates (i.e., the size of the counting target area included in the standard plate), by using the imaging system 1 without changing the current settings, the reliability of the counting function of the imaging system 1 can be ensured.

[0077] On the other hand, for the standard plate 230, it can be confirmed that the imaging system 1 has not exhibited a reliable level of measurement performance. In such a case, the inspector can search for the lighting conditions under which the imaging system 1 exhibits a reliable level of measurement performance for the standard plate 230 by changing the settings of the imaging system 1 to adjust the lighting conditions and further repeating the inspection. Therefore, when counting phase objects of the size corresponding to the standard plate 230, by using the imaging system 1 under the lighting conditions determined through the search, the reliability of the counting function of the imaging system 1 can be ensured.

[0078] In addition, in the case of using, for example, the phase difference observation method or the differential interference observation method, the illumination conditions can be adjusted by changing the position of the optical element arranged on the optical path. Further, in the case of using, for example, the oblique illumination observation method or the dark field observation method, the illumination conditions can be adjusted by changing the positional relationship between the illumination light beam and the pupil. Also, in the case of using, for example, the bright field observation method, the illumination conditions can be adjusted by changing the aperture of the aperture stop.

[0079] As described above, according to the imaging system 1, by checking the counting function using the Figure 4 checking method shown, the illumination conditions for ensuring the reliability of the counting function can be searched. Therefore, the user can use the imaging system 1 under appropriate illumination conditions, and the reliability of the counting function of the imaging system 1 can be ensured.

[0080] In the case where the imaging system 1 fails to exhibit a reliable level of counting performance, the illumination conditions of the imaging system 1 can be adjusted, but information indicating what to specifically adjust to more likely improve the illumination conditions can also be displayed to the inspector. The parameters to be changed vary depending on the observation method, as described above. In this case, the inspector can easily grasp what is better to adjust.

[0081] When a researcher or the like who is not familiar with the structure of the imaging device conducts the inspection of the counting function under the above-described inspection method, the adjustment items of the illumination conditions that can be adjusted by the inspector himself / herself are limited. In addition, even if the inspector is familiar with the device structure, there are cases where adjustment is impossible due to physical constraints. Specifically, there are cases where the illumination conditions cannot be adjusted without using a dedicated tool. Therefore, it is also possible to notify the inspector by indicating whether the adjustment of the illumination conditions can be performed on the spot or cannot be handled without sending it to the factory. In this case, the inspector can appropriately judge whether he / she can change the illumination conditions by himself / herself.

[0082] In the above-described embodiment, a standard plate having a structure in which a plurality of microlenses are formed on a transparent flat plate is exemplified, but the structure of the standard plate is not limited to this example. For example, Figure 12 and Figure 13 the standard plates shown can also be used to check the counting function.

[0083] Figure 12 is a schematic cross-sectional view showing another example of the standard plate for counting function inspection. Figure 12The standard plate 250 shown is a standard plate for checking the counting function of the imaging system 1 that captures phase objects, and has a structure in which a plurality of microlenses are formed on a transparent flat plate, which is the same as that of the standard plate 210. The difference between the standard plate 250 and the standard plate 210 is that the microlenses are formed as concave lenses instead of convex lenses. The standard plate 250 also includes a base region 211 that is a portion exposed on the flat plate surface and a plurality of counting target regions 213 that are portions where microlenses are formed on the flat plate. Therefore, the thickness D3 of the counting target region 213 is different from the thickness D1 of the base region 211, and as a result, the phase amount in the thickness direction is also different between the base region 211 and the counting target regions 213. Even when checking the counting function using the standard plate 250, the reliability of the counting function of the imaging system 1 can be ensured in the same manner as when using the standard plate 210.

[0084] Figure 13 It is a schematic cross-sectional view showing another example of the standard plate for counting function inspection. Figure 13 The standard plate 260 shown is a standard plate for checking the counting function of the imaging system 1 that captures phase objects. The standard plate 260 includes a base region 211 and a plurality of counting target regions 214 whose phase amount in the thickness direction is different from that of the base region 211, which is the same as that of the standard plate 210. The difference between the standard plate 260 and the standard plate 210 is that the base region 211 and the counting target regions 214 have different refractive indices, and as a result, even for the same thickness, they have different phase amounts in the thickness direction. Even when checking the counting function using the standard plate 260, the reliability of the counting function of the imaging system 1 can be ensured in the same manner as when using the standard plate 210.

[0085] In the above embodiment, a standard plate in which a plurality of counting target regions are arranged in a square lattice pattern is illustrated, but the arrangement of the plurality of counting target regions is not limited to this example. For example, it is also possible to use Figure 14 and Figure 15 the shown standard plate to check the counting function.

[0086] Figure 14 It is a schematic top view showing another example of the standard plate for counting function inspection. Figure 14The standard plate 270 shown is a standard plate for checking the counting function of the imaging system 1 for photographing a phase object, and has a structure in which a plurality of microlenses are formed on a transparent flat plate, which is the same as that of the standard plate 210. The difference between the standard plate 270 and the standard plate 210 is that the plurality of counting target areas 212 are arranged in a hexagonal lattice instead of a square lattice. As long as a plurality of counting target areas 212 are regularly arranged, the number of counting target areas 212 located in the field of view can be determined. Therefore, even when checking the counting function using the standard plate 270, the reliability of the counting function of the imaging system 1 can be ensured in the same manner as when using the standard plate 210.

[0087] Figure 15 It is a top view schematic diagram showing another example of a standard plate for counting function inspection. Figure 15 The standard plate 280 shown is a standard plate for checking the counting function of the imaging system 1 for photographing a phase object. Similar to the standard plate 270, it is a standard plate in which a plurality of counting target areas (counting target area 212a, counting target area 212b) are arranged in a hexagonal lattice. The difference between the standard plate 280 and the standard plate 270 is that the sizes of the plurality of counting target areas are different, including the counting target area 212a and the counting target area 212b with different sizes. As long as the learned model corresponds to the sizes of the counting target area 212a and the counting target area 212b, even when checking the counting function using the standard plate 280, the reliability of the counting function of the imaging system 1 can be ensured in the same manner as when using the standard plate 210. In addition, an example in which there are two sizes of counting target areas is shown, but the number of sizes of counting target areas can also be three or more.

[0088] Figure 16 It is a top view schematic diagram showing another example of a standard plate for counting function inspection. Figure 16 The standard plate 290 shown is a standard plate for checking the counting function of the imaging system 1 for photographing a phase object, and is a standard plate in which a plurality of counting target areas 212a and a plurality of areas outside the counting target 212c are arranged in a hexagonal lattice. The counting target area 212a is formed to be approximately the same size as the size of the cells to be counted in actual observation. In contrast, the area outside the counting target 212c is formed to be approximately the same size as the size of minute structures such as dust that should not be counted. Even when using the standard plate 290, the reliability of the counting function of the imaging system 1 can be ensured. In particular, by checking the counting function using the standard plate 290, it is possible to simultaneously check whether the learned model is appropriate, together with whether the settings of the imaging device 10 such as the lighting conditions are appropriate, that is, whether the learned model can appropriately count only the correct objects without erroneously counting dust and the like.

[0089] In the above-described embodiments, an example in which a plurality of regions of objects to be counted are regularly arranged is shown. However, as long as the number of regions of objects to be counted in the field of view can be determined, the plurality of regions of objects to be counted do not necessarily need to be regularly arranged. For example, the Figure 17 standard plate shown can also be used to check the counting function.

[0090] Figure 17 is a top view schematic diagram showing another example of a standard plate for counting function inspection. Figure 17 The standard plate 295 shown is a standard plate for checking the counting function of the imaging system 1 for photographing phase objects. The standard plate 295 is different from the standard plate 210 in that it includes a plurality of regions of objects to be counted 212 arranged irregularly and positioning marks M. Further, in the standard plate 295, the number of regions of objects to be counted 212 located in the field of view when the positioning mark M is aligned with a predetermined position of the field of view, such as the upper left corner of the field of view, is determined in advance for each imaging magnification. Therefore, even when using the standard plate 295, the imaging device 10 can photograph the standard plate 295 under the condition of determining the number of regions of objects to be counted located in the field of view of the imaging system 1. This condition includes the case where the standard plate 295 is arranged at a predetermined position with respect to the field of view and the case where the imaging magnification of the imaging system 1 is known. Therefore, even when using the standard plate 295, the reliability of the counting function of the imaging system 1 can be ensured.

[0091] The above-described embodiments show specific examples for easily understanding the invention, but the embodiments of the present invention are not limited to these specific examples. The inspection method, computer-readable recording medium, and standard plate can be variously deformed and changed without departing from the scope of the claims.

[0092] In the above-described embodiments, an example in which the regions of objects to be counted are formed using microlenses is shown, but the shape of the microlenses is not particularly limited. The lens shape can be spherical or aspherical. Further, the lens shape is not limited to an isotropic shape, and can also be a non-isotropic shape such as a cylindrical lens. However, if the lens shape is isotropic, the counting result does not depend on the illumination direction, and thus it is preferable in that it does not impose a restriction on the orientation of the standard plate during inspection. Further, a shape having irregularities is also preferable in that it is likely to have a shadow and is likely to visually confirm the illumination conditions particularly when using the oblique observation method.

[0093] The region of the object to be counted only needs to be a region having a phase amount different from that of the base region in the thickness direction, that is, a region having a different optical path length. Therefore, the region of the object to be counted can also include, for example, any imaging phantom that can be used as a biological sample imitating cells or tissues, instead of the microlenses.

[0094] The above inspection method can be performed at any timing, but it is preferably performed at the factory before the imaging system 1 is shipped, and preferably performed at the site where the imaging system 1 is used after shipment. Thus, the reliability of the counting function of the delivered imaging system 1 can be assured to the user of the imaging system 1.

[0095] In the above embodiment, an example in which the standard plate is made of an optical material is shown, but any material capable of designing the phase amount may be used, and it is not limited to an optical material. However, since it is a reference for evaluating the reliability of the counting function, the material of the standard plate is preferably a material that is not easily deformed and whose characteristics do not easily deteriorate.

[0096] In the above embodiment, the imaging system 1 in which the imaging device 10 is disposed in the incubator 20 and used is illustrated, but the imaging device 10 of the imaging system is not limited to being used in the incubator. The imaging device 10 of the imaging system may also be used in a working space such as a clean bench. The imaging system 1 may also be, for example, Figure 18 the general microscope system 2 shown.

Claims

1. A method for inspecting a camera system, characterized in that: using an image sensor of the camera system to photograph a standard plate, wherein, the standard plate includes a substrate region and a plurality of counting object regions regularly arranged on the substrate region, the phase amount in the thickness direction of the plurality of counting object regions is different from that of the substrate region, and a known number of counting object regions among the plurality of counting object regions are arranged within the field of view of the image sensor of the camera system; using a processor of the camera system to execute a counting function for counting the counting object regions included in the image of the photographed standard plate; using the processor to compare the number of the counting object regions included in the image of the standard plate with the known number of the counting object regions arranged within the field of view of the image sensor of the camera system; and using the processor to determine the reliability of the counting function according to the result of the comparison.

2. The inspection method according to claim 1, characterized in that, in the inspection method, an evaluation result of the counting function based at least on the counting result of the counting object regions is further output.

3. The inspection method according to claim 2, characterized in that, the evaluation result includes a first evaluation result related to the accuracy of counting based on the counting result and the known number.

4. The inspection method according to claim 2, characterized in that, the evaluation result includes a second evaluation result related to the precision of counting based on the counting result.

5. The inspection method according to claim 3, characterized in that, the evaluation result includes a second evaluation result related to the precision of counting based on the counting result.

6. The inspection method according to claim 1, characterized in that, the photographic magnification of the camera system is known.

7. The inspection method according to claim 2, characterized in that, the photographic magnification of the camera system is known.

8. The inspection method according to claim 3, characterized in that, the photographic magnification of the camera system is known.

9. The inspection method according to claim 4, characterized in that, the photographic magnification of the camera system is known.

10. The inspection method according to claim 5, characterized in that, the photographic magnification of the camera system is known.

11. The inspection method according to claim 1, characterized in that, the standard plate is arranged at a specified position relative to the field of view; and the photographic magnification of the camera system is known.

12. The inspection method according to claim 2, characterized in that, the standard plate is arranged at a specified position relative to the field of view; and the photographic magnification of the camera system is known.

13. The inspection method according to claim 1, characterized in that, the thickness of each of the plurality of counting object regions is different from that of the substrate region.

14. The inspection method according to claim 13, characterized in that, the plurality of counting object regions respectively include microlenses.

15. The inspection method according to claim 1, characterized in that, The refractive index of each of the plurality of regions of the object to be counted is different from that of the substrate region.

16. The inspection method according to claim 1, wherein, the imaging system uses any one of a phase difference observation method, a differential interference observation method, an oblique illumination observation method, a bright field observation method, and a dark field observation method to photograph the standard plate using the image sensor.

17. The inspection method according to claim 16, wherein, the processor switches an algorithm for counting the regions of the object to be counted included in the image according to the sizes of the plurality of regions of the object to be counted.

18. The inspection method according to claim 1, wherein, the substrate region is formed of a phase object, the plurality of regions of the object to be counted are formed of phase objects.

19. A computer-readable recording medium that records a program for inspecting an imaging system, wherein, the program causes a computer of the imaging system to perform the following processing: photographing a standard plate using an image sensor of the imaging system, wherein the standard plate includes a substrate region and a plurality of regions of the object to be counted regularly arranged on the substrate region, a phase amount in a thickness direction of the plurality of regions of the object to be counted is different from that of the substrate region, and a known number of regions of the object to be counted among the plurality of regions of the object to be counted are arranged within a field of view of the image sensor of the imaging system; performing a counting function of counting the regions of the object to be counted included in the image of the photographed standard plate; comparing the number of the regions of the object to be counted included in the image of the standard plate with the known number of the regions of the object to be counted arranged within the field of view of the image sensor of the imaging system; and determining the reliability of the counting function based on a result of the comparison.

Images