Program, storage device, and analysis device

Abstract

To increase precision and speed of analysis / determination while maintaining an optical system.SOLUTION: A program allows a computer to execute an input step of inputting light signal information acquired from an observation object by a ghost cytometry method, a determination step of determining an observation object from light signal information, and an output step of outputting a determined result.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a program, a storage device, and an analysis device. This application claims priority to Japanese Patent Application No. 2015-212356, filed in Japan on October 28, 2015, and U.S. Provisional Patent Application No. 62 / 372,321, filed in the United States on August 9, 2016, the contents of which are incorporated herein by reference. [Background technology]

[0002] For example, Japanese Patent No. 5534214 (Patent Document 1) discloses a flow cytometer and a flow cytometry method using the flow cytometer.

[0003] Next-generation flow cytometry technology is an analytical technique that combines advanced imaging and molecular spectroscopy technologies with flow cytometry, and is making remarkable progress. Previously, cell characteristics could only be evaluated using limited information such as total fluorescent brightness. Next-generation flow cytometry technology evaluates cells using spatial information including fluorescent cell images and molecular spectra, improving the quality of single-cell analysis while maintaining the throughput of conventional flow cytometry, dramatically increasing the amount of information generated from single-cell analysis. However, conventional methods have a problem in that they cannot effectively utilize the vast amount of information generated by, for example, single-cell image cytometry. Specifically, (1) the explosive amount of multidimensional big data generated by next-generation cytometry from diverse and complex cell populations cannot be effectively recognized, processed, and utilized with limited human knowledge and capabilities; (2) the discrimination and cell separation that go through "image" reconstruction processing for high-speed cell sorting takes too much time and cost; and (3) optical imaging methods cannot be optimized for each subject or measurement purpose, resulting in the generation and processing of unnecessary information, which limits the speed, sensitivity, and accuracy of cytometry as a whole. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5534214 Summary of the Invention [Problem to be solved by the invention]

[0005] Analytical devices such as flow cytometers are required to analyze and distinguish objects quickly and accurately. However, the practical application of next-generation flow cytometry faces the following challenges: (1) Humans cannot effectively recognize, process, and utilize the explosive amount of multidimensional big data generated by next-generation cytometry from diverse and complex cell groups with limited knowledge and ability; (2) discrimination and cell separation through "image reconstruction" and "molecular spectrum display in frequency space by Fourier transform" processing for high-speed cell sorting is too time-consuming and costly; and (3) optical imaging methods cannot be optimized for each object and measurement purpose, resulting in the generation and processing of unnecessary information, which limits the improvement of the speed, sensitivity, and accuracy of the entire cytometry. Therefore, the present invention aims to provide an analytical device that improves the accuracy and speed of analysis and discrimination while maintaining the optical system, and an analytical device that improves the accuracy and speed of analysis and discrimination by quickly optimizing the optical system or detection system, such as the light irradiation area. [Means for solving the problem]

[0006] The present invention is based on the knowledge that it is possible to provide an analytical device that can quickly and accurately analyze and discriminate an observed object by analyzing based on signals such as light and electromagnetic waves from the observed object without imaging, and optimizing the light source system or detection system using machine learning, as well as optimizing the methods for analyzing and discriminating the observed object using machine learning.

[0007] By entrusting each key aspect of single-cell flow cytometry to machine learning, it is possible to measure, analyze, and discriminate big data of cell information intelligently. We will solve the above problems by realizing (1) a cell classification method that is not bound by human bias, (2) a high-speed cell spatial information capture and analysis method that does not take cell "images," and (3) an optical capture method that is automatically optimized according to the subject.

[0008] First, in cell classification (1), a large amount of cell information, including cell morphology, nuclear morphology, molecular localization, and molecular information, is phenotyped by machine learning to achieve optimal classification without human bias. On the other hand, the classification results of the machine can be evaluated interactively, for example by interpreting them from a human, biological, or genetic perspective and teaching the machine. By teaching the machine, it is also possible to improve its sensitivity to specific target cells.

[0009] When capturing and analyzing spatial information (2), the time waveform signal obtained in the process of high-speed imaging using a single pixel detection element contains compressed spatial information of the subject (although it cannot be recognized by the human eye). Learning this one-dimensional time waveform data into a machine is equivalent to learning a two-dimensional image, and dramatically improves the processing speed without compromising the quality of the information. In particular, the inventors have developed a high-speed, high-sensitivity (fluorescence) imaging technology using a single pixel detection element, utilizing an optical structure named Ghost Motion Imaging (GMI) and the relative movement of the subject. The time waveform produced by GMI has an imaging mechanism that compresses spatial information into a time waveform, and by directly performing machine learning on this time waveform, cell analysis can be performed with the same high accuracy and sensitivity as analyzing a reconstructed image without reconstructing the cell spatial information as an "image", making it possible to significantly speed up analysis and discrimination. Cell classification using machine learning of GMI time waveforms in this method can obtain results equivalent to cell classification using machine learning on GMI reconstructed images, and therefore can also be used as a means to solve (1).

[0010] In optimizing the optical imaging method according to the target (3), for example, by performing machine learning between GMI images and conventional camera images, for example, by dynamically modifying the optical structure used by GMI, it is possible to realize high speed, high sensitivity, and high accuracy of image cytometry. In addition, by feeding back the cytometry results and evaluation to the optical imaging method, it is also possible to increase the sensitivity of the cytometry target.

[0011] A first aspect of the present invention relates to an analysis method using the following analysis device: Light source 1 and a light irradiation area 3 where light from a light source 1 is irradiated; A light receiving unit 7 that receives scattered light (including Raman scattering), transmitted light, fluorescent light, or electromagnetic waves from an observation target 5 present in the light irradiation area 3 and converts them into an electrical signal; A memory unit 9 that receives and records the electrical signal from the light receiving unit 7; an analysis unit 11 that analyzes the electrical signals related to scattered light, transmitted light, fluorescence, or electromagnetic waves recorded by the memory unit 9 and records the analysis results; an optical system control unit (13) that optimizes the light source (1) or the light irradiation area (3) based on the analysis results; has.

[0012] This analysis device preferably optimizes the discrimination algorithm of the analysis unit 11 by machine learning.

[0013] In this analysis device, preferably, the analysis unit 11 analyzes the object of observation using scattered light, transmitted light, fluorescent light, or electrical signals related to electromagnetic waves without reconstructing an image of the object of observation.

[0014] In this analysis device, preferably, the optical system control unit 13 optimizes the light source 1 by machine learning.

[0015] In this analysis device, preferably, the light from the light source 1 has a plurality of light regions 21 , and the optical system control unit 13 controls the optical structure of the plurality of light regions 21 .

[0016] In this analysis device, preferably, the optical system control unit 13 analyzes the area where the observation target 3 exists based on the electrical signal, and controls so as to limit the light irradiation area 3.

[0017] In this analysis device, preferably, the optical system control unit 13 analyzes the density of the observed object 5 based on the electrical signal to obtain density information of the observed object, and controls the light source 1 or the light irradiation area 3 based on the density information.

[0018] This analytical device preferably further includes a light-receiving system control unit 27 that receives an electrical signal from the light-receiving unit 7 and optimizes a light-receiving area 25, which is an area where light is irradiated to the light-receiving unit 7. In this analytical device, the light-receiving system control unit 27 preferably optimizes the light-receiving area 25 by machine learning.

[0019] A preferred embodiment of this analytical device is a flow cytometer having any of the above-mentioned analytical devices. This flow cytometer has a flow cell 31 including a light irradiation region 3.

[0020] This flow cytometer preferably has a classification section 33 that distinguishes the observation target based on the analysis results of the analysis section 11 and classifies the observation target 5 . Effect of the Invention

[0021] According to the present invention, it is possible to provide an analysis method that can improve the accuracy and speed of analysis and discrimination while maintaining the optical system. Also, according to the present invention, it is possible to provide an analysis method that can improve the accuracy and speed of analysis and discrimination by quickly optimizing the optical system or detection system such as the light irradiation area. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 is a conceptual diagram showing an overview of the analytical device of the present invention. [Diagram 2] Figure 2 is a photograph, instead of a drawing, showing the results of classifying GMI time-series single-cell signals generated from single-cell images using machine learning. [Diagram 3]FIG. 3 is a photograph, instead of a drawing, showing the results of classifying GMI time-series single-cell signals generated from single-cell images using machine learning. [Figure 4] Figure 4 shows an example of optimizing the optical structure while maintaining discrimination accuracy and image accuracy by comparing the GMI image of the observed object with the normal camera image. [Diagram 5] FIG. 5 is a conceptual diagram showing an object being observed passing through patterned illumination. [Figure 6] FIG. 6 is a conceptual diagram showing the state of fluorescence emitted by the object to be observed shown in FIG. [Figure 7] FIG. 7 is a conceptual diagram showing a time-series detection signal when the fluorescence emitted by the object of observation shown in FIG. 5 is detected. [Figure 8] FIG. 8 is a conceptual diagram showing the positions and fluorescence intensities of fluorescent molecules determined from the intensity of the detection signal, that is, a fluorescent image. [Figure 9] FIG. 9 is a diagram for explaining the principle of image reproduction. [Figure 10] FIG. 10 is a diagram for explaining an example of an image reproduction process. [Figure 11] FIG. 11 is a conceptual diagram showing an example of a process for optimizing a lighting pattern. [Figure 12] FIG. 12 is a set of photographs, instead of drawings, showing examples of optimized lighting patterns for various actual observation objects. [Figure 13] FIG. 13 is a conceptual diagram showing an example of a process for optimizing an illumination pattern. [Figure 14] FIG. 14 is a conceptual diagram showing an example of a process for optimizing an illumination pattern. [Figure 15] FIG. 15 is a conceptual diagram showing an example of a process for optimizing an illumination pattern. [Figure 16] FIG. 16 is a diagram showing target samples for learning in the embodiment. [Figure 17] FIG. 17 is a diagram showing an optical structure in an embodiment. [Figure 18] FIG. 18 is a diagram showing an optical structure in an embodiment. [Figure 19]FIG. 19 shows a large number of single-cell image group samples in an embodiment. [Figure 20] FIG. 20 is a diagram showing an optical structure in an embodiment. [Figure 21] FIG. 21 is a diagram showing a group of virtual flow cytometry cell time waveforms in an example. [Figure 22] FIG. 22 shows a group of cells classified in an embodiment (lower right), the average signal of the time-series signals generated by the group of cells (top), and a cell image reconstructed based on the average signal. [Diagram 23] FIG. 23 shows a group of cells classified in an embodiment (lower right), the average signal of the time-series signals generated by the group of cells (top), and a cell image reconstructed based on the average signal. [Figure 24] FIG. 24 is a diagram showing the optical structures before and after optimization and the effect of optimization on discrimination accuracy in an embodiment. [Diagram 25] 1 is an example showing an example of an analysis system. [Figure 26] FIG. 13 is a diagram illustrating an example of a discriminant calculation circuit of the analysis unit. [Figure 27] FIG. 1 is a diagram showing conventional cytometry. [Figure 28] FIG. 1 is a diagram showing an example of a processing method for observing an observation object in the GMI method. [Figure 29] FIG. 1 is a diagram illustrating the concept of the GMI method. [Diagram 30] FIG. 1 shows examples of three types of cells specifically used. [Diagram 31] FIG. 13 is a diagram showing an example of a method for forming a discriminator. [Diagram 32] FIG. 1 shows an example of the results of discriminating different cell types. [Diagram 33] FIG. 1 illustrates an example of a fluidics system in a flow cytometry implementation. [Diagram 34] FIG. 13 is a diagram showing an example of discrimination accuracy in an ideal fluid experiment system. [Diagram 35] FIG. 11 is a diagram illustrating an example of discrimination accuracy in a robust discrimination method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, the embodiments for carrying out the present invention will be described with reference to the drawings. The present invention is not limited to the embodiments described below, and includes appropriate modifications of the embodiments described below within the scope obvious to those skilled in the art.

[0024] An analytical device according to a first aspect of the present invention is shown in Fig. 1. As shown in Fig. 1, this analytical device includes a light source 1, a light irradiation area 3 to which light from the light source 1 is irradiated, a light receiving unit 7 that receives scattered light (including Raman scattering), transmitted light, fluorescence, or electromagnetic waves from an observation object 5 present in the light irradiation area 3 and converts them into electrical signals, a memory unit 9 that receives and records the electrical signals from the light receiving unit 7, an analysis unit 11 that analyzes the electrical signals related to the scattered light, transmitted light, fluorescence, or electromagnetic waves recorded by the memory unit 9 and records the analysis results, and an optical system control unit 13 that optimizes the light irradiation area 3 by machine learning or the like based on the analysis results.

[0025] This analytical device preferably further includes a light receiving system control unit 27 that receives an electrical signal from the light receiving unit 7 and optimizes the light receiving area 25, which is an area where light is irradiated to the light receiving unit 7. In this analytical device, the light receiving system control unit 27 preferably optimizes the light receiving area 25 by machine learning. Note that an analytical device that does not include the optical system control unit 13 and only has the light receiving system control unit 27 can perform rapid and accurate analysis. Such an analytical device includes a light source 1, a light irradiation area 3 irradiated with light from the light source 1, a light receiving unit 7 that receives scattered light, transmitted light, fluorescence, or electromagnetic waves from an observation target 5 present in the light irradiation area 3 and converts them into electrical signals, a memory unit 9 that receives and records electrical signals from the light receiving unit 7, an analytical unit 11 that analyzes the electrical signals related to the scattered light, transmitted light, fluorescence, or electromagnetic waves recorded by the memory unit 9 and records the analysis results, and a light receiving system control unit 27 that receives electrical signals from the light receiving unit 7 and optimizes the light receiving area 25, which is an area where light is irradiated to the light receiving unit 7.

[0026] The present invention will now be outlined.

[0027] Highly accurate and sensitive cell classification method free from human bias By phenotyping large amounts of cell information, including cell morphology, nuclear morphology, molecular localization, and molecular information, generated by advanced optical technology using machine learning, we can perform optimal cell classification that is objective and accurate, with minimal human bias. On the other hand, the classification results of the machine can be evaluated interactively, such as by interpreting them from a human, biological, or genetic perspective and teaching the machine. By teaching the machine, it is also possible to improve its sensitivity to specific target cells.

[0028] Figure 2 shows the results of classifying GMI time-series single-cell signals generated from single-cell images using machine learning. For example, for GMI time-series cell signals generated by a single cell group in a complex and large-volume sample (e.g. blood), first classification is performed using unsupervised machine learning (cell images classified into similar groups, Figure 2, left)), and typical representative examples of each group (template images, Figure 2, right) are presented to a human. Then, based on the templates, the human selects cells that fit the purpose (teaching the machine). Based on the results, the machine modifies its judgment criteria, enabling it to classify target cells with higher sensitivity and speed.

[0029] Figure 3 shows the classification results of GMI time series 1 cell signals generated from 1 cell images by machine learning. Since the images were taken randomly, they may not necessarily be single cells. In each group, the left figure shows the original cell image of the GMI time series 1 cell signal belonging to the same classification, and the right figure shows a template cell image representative of the same classification generated by GMI reconstruction processing from the average signal of the GMI time series 1 cell signal belonging to the same classification. It is also possible to distinguish whether there is one cell or multiple cells, and when considering application to flow cytometry, this is an effective approach in reducing the false positive rate. For example, in current flow cytometry, which distinguishes cells by measuring the total amount of fluorescent molecules, the main false positive results are whether there is one cell or multiple cells, or whether there is a foreign object mixed in. The present invention can provide additional spatial information to existing flow cytometry information, reducing the false positive rate. In addition, the generation of a template image representative of the same classification is useful in practice because it allows you to check whether the classification by machine learning is in accordance with the user's intention.

[0030] The introduction of machine learning mentioned above includes not only the fusion of flow cytometry technology with high-speed imaging technology such as GMI, but also the fusion of flow cytometry technology with nonlinear molecular spectroscopy technology (Raman spectroscopy, stimulated Raman scattering spectroscopy, coherent anti-Stokes Raman scattering spectroscopy). In this case, instead of images or time waveforms, the time series signal of the scattering spectrum is machine-learned, greatly shortening the analysis time without going through Fourier transform, and classification without human bias can be performed. Interactive systems can also be constructed. For example, cell classification can be performed by unsupervised machine learning, a template spectrum can be generated, and a human can educate by looking at this template, and more accurate and purposeful cell classification can be performed based on supervised machine learning.

[0031] A high-speed imaging and analysis method to obtain spatial information of cells without taking cell images For example, in the GMI method, the time waveform signal obtained in the high-speed imaging technique using a single pixel detection element contains effectively compressed spatial information of the imaging subject (although it cannot be recognized by the human eye). Learning this one-dimensional time waveform data into a machine is equivalent to learning a two-dimensional image. Therefore, while the processing speed can be dramatically improved by not going through the image reconstruction process, the information and discrimination accuracy do not decrease. It is possible to process cell spatial information quickly, highly accurately, and with high sensitivity without reconstructing an "image". For example, the unsupervised cell classification shown in Figures 2 and 3 also uses compressed time waveform signals. Even in high-precision, high-sensitivity cell classification free from human knowledge bias, compressed time series signals can be used directly without going through the image reconstruction process.

[0032] Automatically optimizes light for each subject For example, by performing machine learning between GMI images and conventional camera images, the optical structure used by GMI can be dynamically modified. Also, the cytometry results (including human judgment such as wanting specific information on the observed object) can be reflected in the optical photography method.

[0033] Introducing automated machine learning into the optical photography process As shown in Figure 4, the characteristics of the object of observation vary greatly depending on the purpose and researcher (e.g. bacteria or blood). By introducing automatic machine learning into the optical imaging process, the required optical structure is automatically optimized using an optical spatial modulator or similar. The optical structure has parameters such as the overall shape (rectangle, diamond, or ellipse), size, density of bright areas in the random pattern, and size of each bright pattern. Machine learning optimizes these parameters to speed up optical imaging and information processing while improving accuracy.

[0034] By introducing machine learning to the observation target of next-generation flow cytometry, which produces large amounts of multidimensional information, the following effects can be obtained: a. Large amounts of multidimensional data can be processed, b. High-speed analysis and discrimination processing, c. High accuracy, d. No interference from human differences or human fatigue, e. Accurate capture of cellular features that could not be recognized by humans with limited knowledge and perception. Not only when capturing spatial information of cells using fluorescence imaging, but also in molecular spectroscopy, it will become possible to perform high-precision analysis and judgments at high speed from spectral data that cannot be distinguished by the human eye.

[0035] By processing time waveform signals that effectively compress spatial information about cells using optical imaging techniques, rather than two-dimensional information such as cell images, it is possible to speed up processing without compromising the quality of the information.

[0036] By optimizing the optical imaging method according to the observation target, it becomes possible to collect target information accurately while effectively compressing it, and it is possible to speed up optical imaging and information processing without compromising cytometry accuracy. In addition, by reflecting the cytometry results (including human judgment, such as wanting specific information on the observation target) in the optical imaging method, it is possible to make corrections according to the purpose and increase the sensitivity of cytometry.

[0037] Each element of the analytical device of the present invention will be described below.

[0038] optical system The light source 1 and the light irradiation area 3 to which the light from the light source 1 is irradiated form an optical system for irradiating the light to the observation object. The optical system may appropriately have optical elements including mirrors, lenses, spatial light modulators, and filters not shown. The optical system may be an optical system (system) having a structured illumination pattern having a plurality of areas with different optical characteristics. An example of the optical system may be a group of optical elements having a light source and a filter for receiving light from the light source and forming a structured illumination pattern. Another example of the optical system is a group of light sources having a plurality of light sources that constitute an illumination pattern (or a group of optical elements including a group of light sources and optical elements).

[0039] The light from the light source is irradiated with the light pattern on the measurement object, for example, by passing through a filter having a pattern of optical characteristics. The light source may be continuous light or pulsed light, but continuous light is preferable. The light source may consist of a single light source, or may include multiple light sources arranged in a regular pattern (for example, including multiple light sources arranged at equal intervals in the vertical and horizontal directions). In this case, it is preferable that either or both of the intensity and wavelength of the multiple light sources can be controlled.

[0040] The light source may be white light or monochromatic light. Examples of light characteristics include, but are not limited to, characteristics (e.g., transmittance) related to one or more of light intensity, light wavelength, and polarization. An example of a structured illumination pattern having a plurality of regions with different light characteristics is one having a plurality of regions with a first light intensity and a plurality of regions with a second light intensity. An example of a plurality of regions with different light characteristics is one having parts with different light characteristics randomly scattered within a certain area. Another example of a plurality of regions with different light characteristics is one having a plurality of regions divided into a lattice shape, and the plurality of regions has at least a region with a first light intensity and a region with a second light intensity. This structured illumination pattern having a plurality of regions with different light characteristics can be achieved, for example, by adjusting the intensity or frequency of each light source included in the plurality of light sources, or can be obtained by irradiating a transparent film with a pattern printed thereon with light from the light source. Preferably, this structured illumination pattern is irradiated onto the observation object.

[0041] For example, when the observation target 5 is mounted on a specific stage or moves on a specific stage, the area of ​​the stage where light from the light source is irradiated is the light irradiation area 3. Usually, the observation target 5 exists in the light irradiation area 3 or passes through the light irradiation area 3.

[0042] Various objects can be observed depending on the application as the observation object 5. Examples of the observation object include, but are not limited to, cells, body fluids, and an eyeball (which may be a moving eyeball).

[0043] Light receiving unit (photographing unit) The light receiving unit 7 is a detection element that receives scattered light (including Raman scattering), transmitted light, fluorescence, or electromagnetic waves (hereinafter also simply referred to as "optical signals") from the observation object 5 present in the light irradiation area 3 and converts them into electrical signals. When electromagnetic waves are received, analysis based on various spectroscopic techniques can be performed. The light receiving unit 7 may include optical elements such as an optical spatial modulator, and may be capable of appropriately adjusting the optical signal from the observation object 5. If the area of ​​the optical signal from the observation object 5 present in the light irradiation area 3 that reaches the light receiving unit 7 is defined as the light receiving area, the light receiving area may be controlled by these optical elements.

[0044] In other words, the light receiving unit 7 may be a structured detection system having a plurality of regions with different optical characteristics. The light receiving unit 7 may be configured to include, as optical elements, an optical spatial modulator or a film partially coated or painted with a substance that changes the transparency, such as aluminum, silver, or lead. In other words, the light receiving unit 7 may be configured by disposing the above-mentioned optical elements between the light receiving unit 7 and the observation target 5 to which uniform illumination is irradiated. If the region of the optical signal from the observation target 5 present in the light irradiation region 3 that reaches the light receiving unit 7 is defined as the light receiving region, the light receiving region may be controlled by these optical elements.

[0045] The light receiving unit 7 has a light receiving device (imaging device) and preferably has one or a few pixel detection elements. Examples of the one or a few pixel detection elements are, but are not limited to, a photomultiplier tube and a multichannel plate photomultiplier tube. The few pixel detection elements are compact and each element can be operated in parallel at high speed, so that they can be preferably used in the present invention. Examples of one pixel detection elements are disclosed in Patent No. 4679507 and Patent No. 3444509. Examples of the light receiving device are a single or a few light receiving elements such as a PMT (photomultiplier tube), a line type PMT element, an APD (Avalanche Photodiode), a Photodetector (PD), or a CCD camera or a CMOS sensor.

[0046] The light receiving unit 7 may have multiple types of detection devices. When image reconstruction is required for optimization, for example, one detection device may be a detection system based on GMI, and the other detection device may be a normal camera. In this case, as shown in Fig. 4, the image derived from GMI and the image derived from the camera are compared, and the optical system and detection system are adjusted so that the difference between the reconstructed image derived from GMI and the image derived from the camera is reduced.

[0047] storage section The memory unit is an element that is connected to elements such as the light receiving unit so as to be able to send and receive information, and records information. When the light receiving unit has a storage device such as a memory or hard disk, it functions as the memory unit. When the light receiving unit is connected to a computer, the computer's storage device (memory, hard disk, etc.) as well as a server connected to the computer function as the memory unit. The memory unit receives and records electrical signals from the light receiving unit 7.

[0048] The analysis unit discriminates and classifies the scattered light, transmitted light, fluorescence, or electrical signals related to electromagnetic waves recorded by the memory unit 9, and records the results. By using machine learning for the analysis, it is possible to discriminate and classify cell data such as GMI compressed time series signals that cannot be read by the human eye. In other words, it is preferable that the analysis unit stores a program for performing machine learning and is capable of performing machine learning on given information. Specific details of the discrimination and classification will be shown in the examples, but the analysis unit, for example, discriminates to what class the observed object belongs. For example, in discrimination using the k-means method, the class in which the distance between the electrical signal pattern that is the template of each class obtained in advance and the newly obtained electrical signal pattern is the smallest is set as the class of this newly obtained electrical signal pattern. In addition, the observed electrical signals are accumulated, and the pattern group of the accumulated electrical signals is classified. For this classification, classification is performed so that each electrical signal pattern from the average electrical signal pattern of each class is minimized. In addition, based on this accumulated data and classification data, a new electrical signal pattern is discriminated. Furthermore, based on the new electrical signal pattern, this accumulated data and classification data are updated as necessary. Updating classification data refers to using new electrical signal patterns to calculate the average data and median data of each class. Updating classification data means, for example, changing the average value of electrical signal patterns a and b, which was previously (a+b) / 2, to (a+b+c) / 3 by adding a new electrical signal pattern c.

[0049] The analysis unit may have, for example, a time series signal information acquisition unit that receives an optical signal for a certain period of time and acquires time series signal information of the optical signal, and a partial signal separation unit that separates partial time series signal information in a partial region of the observation target from the time series signal information. For example, time series signal information in the case where the observation target is a non-cell impurity or the like is accumulated, and when the partial time series signal information of a certain observation target is classified into a pattern that is classified into partial time series signal information of a non-cell impurity or the like, it can be understood that no cells exist in that observation target. This makes it possible to grasp the region where no cells exist without reconstructing an image, which speeds up processing, and by controlling not to irradiate light to this part or not to use time series signal information from this part thereafter, as described below, it is possible to reduce the amount of information, reduce errors, and speed up processing. Depending on the application, the analysis unit may further have a partial image reconstruction unit that extracts or reconstructs information on the image of each part of the observation target (such as emission intensity) from the obtained partial time series signal information of the observation target. The analysis unit may have an image reconstruction unit that reconstructs an image of the observation target using the image of each part of the observation target reconstructed by the partial image reconstruction unit. In this case, it is preferable because the verification work can be carried out by a human, but since the image of the observed object is reconstructed once, it takes time for analysis and judgment.

[0050] The detection signal includes information on the detection intensity for each change over time. The time series signal information acquisition unit acquires the time series signal information of the optical signal. An example of a time series signal information acquisition unit is one that receives a detection signal that is received for a certain period of time by a light receiving unit such as a single or small number of pixel detection element, and detects and stores it as time series signal information. The time series signal information acquired by the time series signal information acquisition unit may be appropriately stored in a storage unit. In addition, the time series signal information acquired by the time series signal information acquisition unit may be sent to the partial signal separation unit to be used for arithmetic processing by the partial signal separation unit.

[0051] The partial signal separation unit is an element for separating partial time-series signal information in a partial region of the observation target from the time-series signal information. The time-series signal information includes detection signals derived from each part of the observation target. For this reason, the partial signal separation unit separates partial time-series signal information, which is time-series signal information in each partial region of the observation target, from the time-series signal information. At this time, the partial signal separation unit reads out information H relating to the stored illumination pattern, and separates the partial time-series signal information using the read information H relating to the illumination pattern and the time-series signal information. In other words, since the time-series signal information has a fluctuation corresponding to the information H relating to the illumination pattern, the time-series signal information can be separated into partial time-series signal information by using the information H relating to the illumination pattern. The partial time-series signal information, which is time-series signal information in each partial region of the observation target, from the time-series signal information may be stored in the storage unit as appropriate. In addition, depending on the application, the partial time-series signal information may be sent to the partial image reconstruction unit for arithmetic processing by the partial image reconstruction unit.

[0052] The partial image reconstruction unit is an element for extracting or reconstructing information regarding the image (emission intensity, etc.) of each part of the observed object from the partial time series signal information. Since the partial time series signal information is time series signal information in each partial region, information f regarding the light intensity in each region can be obtained. Information regarding the image (emission intensity, etc.) of each part of the observed object may be appropriately stored in the storage unit. In addition, information regarding the image (emission intensity, etc.) of each part of the observed object may be sent to the image reconstruction unit for arithmetic processing by the image reconstruction unit. In this case, for example, since the observed object can be analyzed before reconstructing the image, it is possible to quickly optimize the light source system and detection system, and also obtain an image of the observed object.

[0053] The image reconstruction unit is an element that reconstructs an image of the observed object using the images of each part of the observed object reconstructed by the partial image reconstruction unit. Since the images of each part of the observed object are images of the respective areas of the observed object, the image of the observed object can be reconstructed by combining these images.

[0054] This analysis device preferably optimizes the discrimination algorithm of the analysis unit 11 by machine learning. In other words, the analysis unit 11 is equipped with a discrimination algorithm for performing various analyses, and this discrimination algorithm is optimized by machine learning. The discrimination algorithm includes an algorithm for discriminating using the above-mentioned classification of the observed object and classification of the signal when the observed object is not present. Examples of analysis include grasping the optical signal components characteristic of the observed object, setting the threshold value used in the discrimination work, and optimizing conditions for the optical system and detection system.

[0055] Machine learning is well known, as disclosed in, for example, Patent Publication No. 5574407, Patent Publication No. 5464244, and Patent Publication No. 5418386. An example of machine learning is learning using the Ada Boosting algorithm. Machine learning, for example, obtains optical signals for multiple objects among the objects to be observed, and learns the characteristics of the objects to be observed from the obtained optical signals. By performing machine learning, it becomes possible to detect the presence or absence of specific cells extremely efficiently and quickly. The subject of machine learning is not limited to images, and for example, vibrations that are not imaged, such as those using Raman spectroscopy, can be detected and the detected signal can be used as the subject of analysis. For example, the analysis unit can analyze the optical signals of multiple objects to be observed using machine learning, and perform analysis such as classification and discrimination of the optical signals of the objects to be observed.

[0056] In this analysis device, the analysis unit 11 preferably analyzes the observed object without reconstructing an image of the observed object from electrical signals related to scattered light, transmitted light, fluorescence, or electromagnetic waves. The analysis unit 11 analyzes the observed object using, for example, the above-mentioned time series signal information, partial time series signal information, or GMI. In this example, by recognizing patterns, etc. for multiple objects by machine learning and comparing the time series signal information, partial time series signal information, or GMI with the patterns, etc., it is possible to analyze whether the observed object is a specific object or not, and to analyze information about the observed object including the size and position of the observed object.

[0057] Optical system control unit The optical system control unit 13 is an element for optimizing the light source 1 or the light irradiation area 3 based on the analysis results. The control signal (control command) for controlling the light receiving unit 7, the light source 1, or the optical elements constituting the optical system may be obtained in the analysis unit described above, or may be obtained in the optical system control unit 13. When a control command is obtained in the analysis unit, the optical system control unit 13 can optimize the light source system by controlling the light source 1 or the optical elements constituting the optical system according to the control signal (control command) obtained by the analysis unit 3.

[0058] An example of the optical system control unit 13 is a computer connected to a control device that controls the light receiving unit 7, the light source 1, or the optical elements that constitute the optical system so as to be able to send and receive information. A program is installed in this computer so as to be able to perform specific arithmetic processing and input / output. An example of this program is a program for performing machine learning. In an image capturing mechanism such as GMI, it is preferable that the optical system control unit 13 processes the time waveform electrical signal (GMI) as it is, without reconstructing the image of the observed object, based on the electrical signal from the light receiving unit 7. Depending on the application, it may have a function to reconstruct the image of the observed object, in which case the image quality can be verified.

[0059] In a spectrum time waveform recording mechanism such as Raman spectroscopy, the optical system control unit 13 preferably processes the time waveform electrical signal as it is, without Fourier transforming the spectrum and analyzing the molecular spectrum in frequency space, based on the electrical signal from the light receiving unit 7. However, depending on the application, the optical system control unit 13 may analyze the molecular spectrum in frequency space by Fourier transforming the electromagnetic wave spectrum.

[0060] In this analysis device, preferably, the optical system control unit 13 optimizes the light source 1 or the light irradiation area 3 by machine learning. An example of optimizing the light source 1 is adjusting the light intensity of the light source 1. In this analysis device, preferably, the light from the light source 1 has a plurality of light areas 21, and the optical system control unit 13 controls the optical structure of the plurality of light areas 21. In this analysis device, preferably, the optical system control unit 13 analyzes the presence area of ​​the observed object 5 based on an electrical signal, and controls to limit the light irradiation area 3.

[0061] In this analysis device, preferably, the optical system control unit 13 analyzes the density of the observed object 5 based on the electrical signal to obtain density information of the observed object, and controls the light source 1 or the light irradiation area 3 based on the density information.

[0062] Light receiving control section This analysis device preferably further includes a light receiving system control unit 27 that receives an electric signal from the light receiving unit 7 and optimizes the light receiving area 25, which is the area where the light receiving unit 7 is irradiated with light. The light receiving system control unit 27 may be an analysis unit of the light receiving system described above. That is, the analysis unit employs, for example, a program for machine learning, and classifies the light receiving signals of a part of the light receiving unit from which useful information has not been obtained. When the light receiving signals of a certain part of the light receiving unit are classified into this classification, for example, processing is performed so that information from this part is not used in the analysis. In this way, the amount of processing for the analysis can be reduced, and the processing can be performed quickly. In this analysis device, preferably, the light receiving system control unit 27 optimizes the light receiving area 25 by machine learning. In this preferred embodiment, the light source 1 or the light receiving area 25 may be optimized using a method similar to the optimization of the optical system by the optical system control unit 13. An example of the light receiving system control unit 25 is a computer connected to a control device that controls the light receiving unit 7, the light source 1, or the optical elements that constitute the light receiving system or the light receiving area 25 so as to be able to exchange information with the control device. In other words, this specification discloses not only optimization of the optical system and the light receiving system, but also optimization of only the light receiving system.

[0063] The analytical device may include various elements in known analytical devices, such as a relative position control mechanism.

[0064] Next, an example of the operation of the imaging device of the present invention will be described. FIG. 5 is a conceptual diagram showing that the observation object passes through the patterned illumination. As shown in FIG. 5, the observation object 5 is moved by a relative position control mechanism and passes through the patterned illumination in the optical system. The intensity distribution of this patterned illumination light structure is represented by a matrix H(x, y). This observation object 5 has optical spatial information, for example, fluorescent molecules represented by F1 to F4. These fluorescent molecules do not emit fluorescence or emit different intensity of fluorescence depending on the intensity of the light received. That is, in this example, the fluorescent molecule represented by F2 emits fluorescence first, and its emission intensity is affected by the patterned illumination that passes through. The light from the observation object 5 may be collected by a lens or the like as appropriate. Then, the light from the observation object 5 is transmitted to one or a few pixel detection elements. In the example of FIG. 5, the traveling direction of the observation object is the x-axis, and the y-axis is provided perpendicular to the x-axis on the same plane as the x-axis. In this example, F1 and F2 are observed as fluorescence on the same y coordinate, y1 (this will be denoted as H(x,y1)).F3 and F4 are observed as fluorescence on the same y coordinate, y2 (this will be denoted as H(x,y2)).

[0065] FIG. 6 is a conceptual diagram showing the state of fluorescence emitted by the object of observation shown in FIG. 5. As shown in FIG. 6, fluorescence is emitted from each fluorescent molecule, and for example, F1 and F2 experience the same illumination pattern, and therefore are considered to have similar time response patterns or output patterns. On the other hand, the emission intensities of F1 and F2 are considered to be different. For this reason, the emission intensities of F1 and F2 can be approximated as the product of F1 and F2, which are coefficients specific to each luminescent molecule, and H(x, y1), which is the time response pattern common to the coordinate y1. The same is true for F3 and F4.

[0066] Figure 7 is a conceptual diagram showing the detection signal when the fluorescence emitted by the object of observation shown in Figure 5 is detected. This detection signal is observed as the sum signal of the fluorescence signals shown in Figure 6. This signal contains a pattern of time change of multiple intensities H(x,yn). Then, the fluorescence coefficient (fluorescence intensity) at each coordinate and at each coordinate can be obtained from the intensity (G(t)) of this detection signal and H(x,yn).

[0067] Fig. 8 is a conceptual diagram showing the position and fluorescence intensity of fluorescent molecules obtained from the intensity of a detection signal. As shown in Fig. 8, the fluorescence coefficients (fluorescence intensities) F1 to F4 can be obtained from the detection signal G(t).

[0068] The above principle will be explained in more detail. Figure 9 is a diagram for explaining the image reproduction principle. For example, assume that F(1) and F(2) exist as coordinates within the object. Then, at time 1, F(1) is irradiated with light of the first pattern, and F(2) is not irradiated. At time 2, F(1) is irradiated with light of the second pattern, and F(2) is irradiated with light of the first pattern. At time 3, F(1) is not irradiated with light, and F(2) is irradiated with light of the second pattern. Then, the detection signal G(t) is as follows. G(1)=F(1)H(1), G(2)=F(1)H(2)+F(2)H(1), G(3)=F(2)H(2). By solving this, F(1) and F(2) can be analyzed. By using this principle, even if the number of coordinates within the object increases, F(1) to F(n) can be obtained by performing analysis in the same way.

[0069] Next, if the object is two-dimensional, the internal coordinates of the observed object are taken as F(x,y). Meanwhile, the pattern lighting is also assumed to have coordinates. If the internal coordinates of the observed object are taken as n in the x-axis direction and n in the y-axis direction, then the number of unknowns in F(x,y) is n × n. By measuring the signal in the same manner as above and analyzing the obtained signal G(t), F(x,y) (0≦x≦n, 0≦y≦n) can be reconstructed.

[0070] FIG. 10 is a diagram for explaining an example of the image reproduction process. In this example, the image is expressed as f (target position information vector) in a determinant. Patterned illumination is expressed as H(X, y), where X is a time-varying variable. The detected signal intensity is expressed as g (measurement signal vector). These can then be expressed as g = Hf. As shown in FIG. 10, to obtain f, the inverse matrix H-1 of H is multiplied from the left. On the other hand, there are cases where H is too large and the inverse matrix H-1 of H cannot be easily obtained. In such cases, for example, the transposed matrix Ht of H can be used instead of the inverse matrix. Using this relationship, an initial estimate fint of f can be obtained. After that, the image of the observation target can be reproduced by optimizing f using the initial estimate fint of f.

[0071] In other words, FIG. 10 is a diagram for explaining an example of the image reproduction process. In this example, the image is expressed as f (target position information vector) in a determinant. Patterned illumination is expressed as H(X, y), where X is a time-varying variable. The detected signal intensity is expressed as g (measurement signal vector). These can then be expressed as g = Hf. As shown in FIG. 10, to obtain f, it is sufficient to multiply the inverse matrix H-1 of H from the left. On the other hand, there are cases where H is too large and the inverse matrix H-1 of H cannot be easily obtained. In that case, for example, an initial estimate finit of f can be obtained as the result of multiplying the transposed matrix HT of H and g. After that, the image of the observation target can be reproduced by optimizing f using the initial estimate finit of f.

[0072] The shape of the light structure matches the shape of the object being observed In cytometry, the objects to be observed, such as cells, are often spherical. In this case, the overall shape of the light structure does not need to be rectangular. For example, you can change the bright areas of the four corners of the light structure to dark areas and increase the number of dark areas as long as the quality does not decrease. If the quality decreases, you can then add several new points to the four corners and repeat this cycle.

[0073] When imaging cells, there are dense structures such as the cytoplasm and nucleus, and sparse structures such as cell membranes, localized molecules, and specific chromosome labels (FISH method, etc.). In GMI, basically, a sparse optical structure is desirable for a dense target structure, and it is desirable to design a dense optical structure for a sparse target structure. First, the density of the target is determined based on the GMI electrical signal. For example, when the ratio of the time integral value of the target signal to the product of the maximum value of the target signal and the time width of the target signal (time integral value of the target signal / product of the maximum value of the target signal and the time width of the target signal) is above a certain value, the target is dense, and when it is below a certain value, the target is sparse. This value is adjusted depending on the sample and target structure. Based on this judgment, a sparser or denser optical structure is designed. For example, the occupancy rate of random bright areas in the entire structure is increased or decreased to create a new random pattern (the current GMI optical structure uses a DMD and optical mask, and the structure is simply random. Therefore, it is binary, light and dark, and the bright areas are specified as a certain percentage of the whole, and are scattered randomly). Finally, the above cycle is repeated until the ratio (the ratio of the time integral value of the target signal to the product of the maximum value of the target signal and the time width of the target signal) falls within a certain range (depending on the target).

[0074] Target signal S / N strength Depending on the subject, the optical signal intensity may be very weak and the S / N ratio may be poor. If this deteriorates, high-precision cell information processing and cytometry cannot be performed. One of the methods to improve the S / N ratio in GMI is to bin the pixels of the spatial light modulator into multiple pixels and use them as unit pixels in the GMI optical structure. This increases the amount of light per unit pixel in the GMI optical structure, improving the S / N ratio. In addition, in Raman spectroscopy, one of the methods to improve the S / N ratio is to reduce the amount of light irradiated onto areas where the target object does not pass. This reduces the amount of noise light and improves the S / N ratio.

[0075] The simplest binning method is to bin the same number of pixels vertically and horizontally, such as 2*2, 3*3, etc. However, this increases the structure size of the spatial modulator (corresponding to the actual size of the GMI light structure on the sample), and the throughput, amount of information, and quality are degraded.

[0076] Binning is performed in a horizontally long rectangle, for example, 1 vertical x 2 horizontal. In this case, the spatial resolution in GMI is determined by the size of the vertical pixels, so the spatial resolution does not deteriorate. However, the number of horizontal pixels of the spatial modulator (corresponding to the actual horizontal width of the GMI optical structure on the sample) becomes large, and the throughput is sacrificed.

[0077] Binning is performed in a vertically long rectangle, 2 vertical x 1 horizontal. This deteriorates the spatial resolution because the spatial resolution in GMI is determined by the size of the vertical pixels. However, the number of horizontal pixels of the spatial modulator (corresponding to the actual horizontal width of the GMI optical structure on the sample) does not change, so the throughput does not deteriorate.

[0078] Cell imaging involves complex structures such as cytoskeleton structures, virus infection routes, and cell signaling networks. It is difficult to design an optimal optical structure for such complex structures. In this case, the optical structure can be automatically optimized by machine learning based on the GMI electrical signal, for example, to improve the quality of the reconstructed image. The optimization of the optical structure by machine learning achieves optimization including the above examples by setting the objective function, for example, to improve imaging throughput, reduce the amount of calculation and image information of the electrical signal, improve image quality, improve sensitivity to target features (nucleus / cytoplasm ratio, cell size, chromosome condensation image, chromosome number, etc.), improve signal S / N, and improve discrimination accuracy.

[0079] For example, when performing cell imaging using GMI, the optical structure can be optimized using known machine learning and optimization algorithms. Known machine learning and optimization algorithms include evolutionary algorithms and simulated annealing. For example, by setting the minimization of the area of ​​the illumination area, the maximization of image quality and discrimination accuracy, the maximization of S / N, etc. as the objective function of the optimization algorithm, the optical structure can be optimized by machine learning to achieve the above objectives.

[0080] FIG. 11 is a conceptual diagram showing an example of a process for optimizing an illumination pattern. In this example, an optical signal (GMI signal, ghost motion imaging signal, g) is obtained using the imaging device described above, and an image (F) of the observation object 5 is reconstructed. Meanwhile, the imaging unit photographs the same observation object 5, and obtains a photographed image (C) of the observation object. Then, the optical system control unit compares the reconstructed image (F) with the photographed image (C). This comparison may be performed, for example, by adjusting the reconstructed image (F) and the photographed image (C) so that they are the same size, and then summing up the differences in color or intensity contrast contained in each pixel (or summing up the absolute value of the difference, or the square of the difference), and setting the obtained value as the comparison value (ε). Then, the optical system control unit appropriately changes the illumination pattern, and obtains a new comparison value using an observation object that is the same as the previous observation object 5, or an observation object of the same type. In this way, after obtaining multiple combinations of illumination patterns and comparison values, the optimal illumination pattern may be determined, taking into consideration the amount of information contained in the illumination pattern.

[0081] Also, the size of the observed object may be grasped based on the reconstructed image (F), and the illumination pattern may be controlled so as to obtain an illumination pattern according to the grasped size. This can be done, for example, by obtaining reconstructed images of one or more observed objects, analyzing the size of the required illumination area from these, and controlling the illumination pattern to obtain that size. In this case, the optical system control unit controls the optical system so that the illumination pattern becomes the obtained optimized illumination pattern. The light source system adjusts the illumination pattern according to the control signal from the optical system control unit. In this way, it is possible to obtain an optimized illumination pattern.

[0082] Figure 12 is a photograph instead of a drawing showing an example of optimizing the illumination pattern for various observation objects. In this example, first, the image was reconstructed multiple times to grasp the range of the size of the observation object. This can be easily implemented by analyzing the area in which the reconstructed image exists. Then, the size of the illumination pattern that can cover the area required to capture the observation object was determined, and the illumination patterns included in that size were variously changed to obtain a comparison value (ε), and the optimal illumination pattern was found from the comparison value (ε) of each illumination pattern. By using this optimized illumination pattern, the amount of information could be significantly reduced, and the amount of processing required to reconstruct the image thereafter was significantly reduced, making it possible to perform high-speed imaging.

[0083] A preferred example of this imaging device first grasps the size of the object to be observed based on the above method, and adjusts the size of the illumination pattern. Then, the pattern itself in the illumination pattern over the adjusted size (area) is changed to obtain a comparison value (ε) for multiple patterns. Then, the comparison value (ε) is compared to obtain the optimal illumination pattern. The comparison value (ε) may be the sum of the squared values ​​of the difference values ​​of each pixel.

[0084] Fig. 13 is a conceptual diagram showing an example of a process for optimizing an illumination pattern. In this example, the illumination pattern is optimized based on an optical signal (GMI image) without reconstructing an image. This example further includes a detection system that captures an image of an observed object 5, an estimated signal calculation unit that obtains an estimated signal gC that estimates an optical signal detected by one or a small number of pixel detection elements based on an image of the observed object captured by the detection system, and a calculation unit that changes the illumination pattern while comparing the optical signal g detected by one or a small number of pixel detection elements with the estimated signal gC estimated by the estimated signal calculation unit.

[0085] In this example, for example, the detection system photographs the object to be observed and obtains a photographed image. Then, the photographed image is image-analyzed to grasp the composition and tissue of each part. Then, light information corresponding to each composition and tissue is obtained from a table that stores information on the light (e.g., fluorescence) emitted from each composition and tissue when light is irradiated. In this way, it is grasped what kind of optical response occurs when the object to be observed is irradiated with light. Then, the optical signal g can be estimated based on the photographed image. This is the estimated signal gC estimated by the estimated signal calculation unit. The optical signal g has a spectrum as shown in FIG. 7, for example. Then, the second illumination pattern control unit 107 adjusts the relative intensities of the estimated signal gC and the optical signal g to be approximately the same, and further adjusts the corresponding positional relationship between the estimated signal gC and the optical signal g to be correct by auto-matching the shape of the spectrum (adjusting so that the amount of overlap is maximized), and then calculates the evaluation value (ε) of the estimated signal gC and the optical signal g. ε may be, for example, the sum of the difference (or the absolute value of the difference, or the square of the difference) between the relative intensity of the estimated signal gC and the optical signal g per unit time. Alternatively, for example, the estimated signal gC and the optical signal g may be transformed into a new coordinate space, and the difference in the relative intensity or the like in that space may be calculated. The evaluation value (ε) may be calculated based on various illumination patterns, and the illumination pattern may be optimized using the evaluation value (ε).

[0086] FIG. 14 is a conceptual diagram showing an example of a process for optimizing an illumination pattern. This example is used, for example, when information about the observed object (for example, the image pattern obtained by the image reconstruction unit reconstructing the observed object) has already been stored. That is, this example further includes a control unit that changes the illumination pattern using the image (F) of the observed object reconstructed by the image reconstruction unit. In this example, for example, the control unit performs pattern authentication between the reconstructed image (F) of the observed object and the image pattern stored in advance. Since pattern authentication technology is publicly known, pattern authentication can be implemented by installing a publicly known program for pattern authentication in a computer. This example can be effectively used, for example, when selecting an object (passed or rejected) or inspecting the presence or absence of an object. In addition, this example can be used for applications such as automatically measuring the number of cells contained in a specific area. That is, a preferred example of an imaging device further includes an observed object determination unit that uses the image of the observed object reconstructed by the image reconstruction unit to identify the observed object.

[0087] FIG. 15 is a conceptual diagram showing an example of a process for optimizing an illumination pattern. This example is used, for example, when information on an object to be observed (for example, a pattern of an optical signal g) has already been stored. This example further includes a control unit that changes the illumination pattern using an optical signal g detected by one or a small number of pixel detection elements. In this example, for example, the control unit performs pattern authentication between the optical signal g detected by one or a small number of pixel detection elements and the pattern of the optical signal g stored in advance. Since pattern authentication technology is publicly known, pattern authentication can be implemented by installing a publicly known program for pattern authentication in a computer. Alternatively, for example, both signals g may be converted into a new coordinate space, and the difference in relative intensity, etc. within that space may be measured.

[0088] A preferred example of the imaging device further includes a reconstructed image classification unit that classifies the reconstructed images of the observed object using a plurality of images of the observed object reconstructed by the image reconstruction unit. The images (F) classified by the reconstructed image classification unit are used by the control unit and the determination unit.

[0089] A preferred embodiment of this analytical device is a flow cytometer having any of the above-mentioned analytical devices. This flow cytometer has a flow cell including a light irradiation region 3.

[0090] This flow cytometer preferably has a separation section that distinguishes the observation target based on the analysis results of the analysis section 11 and separates the observation target 5. To be more specific, the target can be separated by differentiating the path after passing through the separation section depending on whether the observation target 5 is the target or not.

[0091] In the flow cytometer, the target object may be analyzed in advance, and information indicating the target object (such as a threshold value) may be stored in the memory unit. Also, an object containing a large amount of the target object may be observed, and the discrimination unit may discriminate the object as an observed object, and machine learning may be performed to learn discrimination information regarding the target object. This discrimination information may be, for example, characteristic peaks contained in various spectra.

[0092] When the object to be observed moves through the flow cell and reaches the light irradiation area, the object to be observed is irradiated with light from the light source. The light receiving unit then receives the light signal from the object to be observed. The analysis unit analyzes the light signal. In doing so, the analysis unit reads out the object discrimination information stored in the memory unit and compares it with the light signal to determine whether the observed object is the object of interest. If the analysis unit determines that the observed object is the object of interest, it sends a control signal to the separation unit indicating that the object was the object of interest. The separation unit, upon receiving the control signal, adjusts the path and guides the object to the path of the object of interest. In this way, it is possible to distinguish and separate objects that are the object of interest from objects other than the object of interest.

[0093] In this case, multiple objects to be observed are measured, and the analysis unit can optimize the optical system or detection system as appropriate, allowing for rapid discrimination with appropriate accuracy. EXAMPLES

[0094] The present invention will now be described in detail with reference to examples. <Example 1-1> (Supervised machine learning, uncompressed conversion of images to time-series signals using optical structures, classifier formation and classification)

[0095] The computer used in this embodiment has a 2.8 GHz Intel Core i7 processor and 16 GB of memory. First, as training samples, we prepared a group of face images and non-face images measuring 19 pixels vertically and 19 pixels horizontally, for a total of 1,100 images each: 1,000 for training and 100 for measuring discrimination accuracy (Figure 16, image source: Center for Biological and Computational Learning at MIT).

[0096] In the computer, the above-mentioned images were given noise (S / N=30dB), and the GMI process was virtually executed to pass them through the optical structure to generate a time series signal. The optical structure was the patterned illumination optical structure or detection optical structure in the experimental system, and the optical structure used here was 19 pixels wide and 343 pixels high (Figure 17), and a time series signal was generated for 1x192 pixels (same as the original image). All waveform signals were labeled as face or non-face, and a linear classification type support vector machine method was used to train 1000 waveforms for training, forming a classifier. Then, as samples for measuring the discrimination accuracy, a time series signal was prepared using the same optical structure based on 100 new face and non-face images. The labels were removed from these samples, and the classifier was automatically discriminated, and the correct answer rate of the label (face or non-face) was measured.

[0097] On the other hand, we added noise (S / N=30dB) to the same image samples, labeled them as faces or non-faces, and formed a classifier by training 1,000 training images using the support vector machine method. We then similarly prepared 100 new face and non-face images as samples for measuring classification accuracy. We removed the labels from these samples and had the classifier automatically classify them, measuring the accuracy rate of the labels (face or non-face).

[0098] As a result, the accuracy of discriminating between face and non-face time-series signals using a classifier that had been trained on time-series signal samples was 87% (number of correct answers for faces and non-faces / total number x 100), and the accuracy of discriminating between face and non-face images using a classifier that had been trained on image samples was 82%. These results demonstrate that even when training and discriminating time-series signals generated by passing images through an optical structure, discrimination results can be obtained that are equal to or better than those obtained when training and discriminating original images.

[0099] Moreover, the time required for learning 1000 samples and discriminating 100 samples did not change whether it was an image or a time series signal.

[0100] <Example 1-2> (Supervised machine learning, compressing and converting images into time-series signals using optical structures, forming discriminators and discriminating) The computer used in this embodiment has a 2.8 GHz Intel Core i7 processor and 16 GB memory. First, as learning samples, a group of face images and a group of non-face images each having 19 pixels vertically and 19 pixels horizontally were prepared, with 1000 images for learning and 100 images for measuring the discrimination accuracy, for a total of 1100 images (FIG. 16).

[0101] In the computer, the above-mentioned images were given noise (S / N=30dB), and the GMI process was virtually executed to pass them through the optical structure, generating a time waveform signal (e.g., GMI waveform). The optical structure was a patterned illumination optical structure or detection optical structure in the experimental system, and the optical structure used here was 19 pixels wide and 50 pixels high (Figure 18), which resulted in the generation of 68 pixels of time series signals, which is 81% compressed from the total number of pixels of the original image. All compressed time series signals were labeled as face or non-face, and a classifier was formed by learning 1,000 waveforms for learning using the support vector machine method. Then, as samples for measuring the discrimination accuracy, compressed time series signals were prepared using the same optical structure based on 100 new face and non-face images. The labels were removed from these samples, and the classifier that was formed was made to automatically discriminate, and the correct answer rate of the label (face or non-face) was measured.

[0102] On the other hand, we added noise (S / N=30dB) to the same image samples, labeled them as faces or non-faces, and formed a classifier by training 1,000 training images using a linear classification support vector machine method. We then similarly prepared 100 new face and non-face images as samples for measuring classification accuracy. We removed the labels from these samples and had the classifier automatically classify them, measuring the accuracy rate of the labels (face or non-face).

[0103] As a result, the accuracy of discrimination between compressed time waveform signals of faces and non-faces (number of correct answers for faces and non-faces / total number × 100) by a classifier trained on compressed time waveform signal samples was 75%, and the accuracy of discrimination between faces and non-faces in images by a classifier trained on image samples was 82%. These results demonstrated that the discrimination accuracy by machine learning can be maintained at the same level even through optical image compression through an optical structure.

[0104] The time required to learn 1000 time waveform samples was 399 seconds, and the time required to learn 1000 image samples was 475 seconds. These results show that the time required to learn the same samples can be reduced by 16% in the case of compressed time signals compared to the original images.

[0105] Furthermore, the time required to classify 100 compressed time series samples was 0.0065 seconds, and the time required to classify 100 image samples was 0.0147 seconds. From these results, it was found that, when classifying the same samples, the compressed time signal classification can reduce the time by 56% compared to the original image classification. EXAMPLES

[0106] Example 2 (Cell classification by unsupervised machine learning) The computer used in this embodiment has a 2.8 GHz Intel Core i7 processor and 16 GB of memory. As a sample, a single cell group generated by dispersing a mouse spleen was stained with calcein AM for live cell staining. The fluorescently labeled single cell solution as described above was spread on a glass slide, and a large number of fluorescent images of the single cell group were taken using a fluorescence microscope with an sCMOS camera (Flash 4.0, manufactured by Hamamatsu Photonics). This image data was read in a computer, the position of the single cell was identified using software (Imagej), and the area around the single cell was segmented and cut out at 70 pixels vertically and horizontally, preparing a large number of single cell image group samples (2165 images) (Figure 19). This single cell image group includes images containing single cells of different sizes, as well as multiple cells and objects other than cells.

[0107] In a computer, noise (S / N=30dB) was added to the above image group, and the GMI process of passing the images through an optical structure was virtually executed to generate a time waveform signal (e.g., a GMI waveform), and a virtual flow cytometry cell time waveform group was prepared (Figure 20). The optical structure used here was 70 pixels vertically and 400 pixels horizontally, resulting in a time series signal of 470 pixels (Figure 21). (Meanwhile, noise (S / N=30dB) was added to the same image sample, and a virtual flow cytometry cell image was prepared.)

[0108] Using software (Matlab), unsupervised machine learning classification was performed on the single-cell time waveform samples prepared as described above. Specifically, the k-means method was used to classify the cells into 20 types of cell groups. From the cell time waveform groups that fell into the same classification group, representative (average) time series signals were generated, and cell images were generated based on these time series signals (Figures 22 and 23). As a result, it can be seen that the cells were correctly classified as single cells or multiple cells, as debris, and by cell size. These are among the largest sources of error in conventional cytometry, and it was shown that they could also be correctly distinguished in unsupervised machine learning using compressed time waveform signals through an optical structure. EXAMPLES

[0109] Example 3 (optimization of optical structures using supervised machine learning) In this example, the same sample as that used in Example 1 was used. As the initial light structure, a random structure with 80 pixels vertically and 20 pixels horizontally was prepared, and a time waveform signal group sample was prepared through the image sample of Example 1. As in Example 1, learning was performed using a linear classification type support vector machine method, and the discrimination accuracy (number of correct answers for faces and non-faces / total number x 100) was obtained. This discrimination accuracy was set as the objective function, and the light structure was optimized by machine learning to maximize the objective function (Figure 24). Specifically, a genetic algorithm was used. The number of individuals was 200, the number of generations was 16,000, roulette selection was used for selection, and uniform crossover was used for crossover. As a result, as in Example 1, when an evaluation was performed using an image sample not used for optimization, the discrimination accuracy was 65% with the initial random light structure, and the discrimination accuracy was 75% with the optimized light structure, indicating an improvement in discrimination accuracy of 10%.

[0110] Next, an example of the analysis unit 11 will be described with reference to FIG. FIG. 25 is a diagram showing an example of an analysis system 200. As shown in FIG. The analysis system 200 includes a flow cytometer 300, an analysis unit 11, and a computer 400. The flow cytometer 300 observes and classifies the observation object 5. The flow cytometer 300 outputs an optical signal related to the observed observation object 5 to the analysis unit 11. The analysis unit 11 discriminates the observation object 5 based on the optical signal related to the observation object 5 from the flow cytometer 300. The computer 400 performs machine learning on the optical signal related to the observation object 5 observed by the flow cytometer 300. The computer 400 changes the discrimination algorithm of the discrimination unit 101 based on the results of the machine learning.

[0111] The flow cytometer 300 includes a light receiving unit 7 and a sorting unit 33 . The light receiving unit 7 receives an optical signal from the observation target 5 and converts it into an electrical signal ES. The light receiving unit 7 outputs the converted electrical signal ES to the signal input unit 100. The classification unit 33 classifies the observed object 5 based on the signal discrimination result R indicating the result of the analysis unit 11 analyzing the electrical signal ES.

[0112] The computer 400 includes a memory unit 9 and a machine learning unit 401 . An input signal SS is stored in the storage unit 9. A machine learning unit 401 performs machine learning on the optical signal stored in the storage unit 9.

[0113] In this example, the analysis unit 11 includes a signal input unit 100 and a discrimination unit 101. The discrimination unit 101 includes a logic circuit whose logic circuit configuration is changeable. The logic circuit may be a programmable logic device such as an FPGA (field-programmable gate array) or an ASIC (application specific integrated circuit).

[0114] The machine learning unit 401 changes the discrimination algorithm of the discrimination unit 101 included in the analysis unit 11 based on the machine learning result. In this example, the machine learning unit 401 changes the logic circuit configuration of the discrimination unit 101 based on the machine learning result. Specifically, the machine learning unit 401 configures discrimination logic LP, which is the logic circuit configuration of a discrimination algorithm suitable for the observed object 5, based on the machine learning result, and changes the logic circuit.

[0115] The signal input unit 100 acquires an electrical signal ES from the light receiving unit 7. The signal input unit 100 outputs the electrical signal ES acquired from the light receiving unit 7 to the storage unit 9 and the discrimination unit 101 as an input signal SS.

[0116] The signal input unit 100 may remove noise from the electrical signal ES by applying a filter to the electrical signal ES. Examples of noise include high-frequency noise and shot noise. By removing the noise from the electrical signal ES, the signal input unit 100 can stabilize a trigger position at which acquisition of the electrical signal ES as the input signal SS begins. By stabilizing the trigger position, the signal input unit 100 can output a signal suitable for use in machine learning as the input signal SS. In addition, the signal input unit 100 may distinguish whether the observed object 5 is one cell or multiple cells, whether the observed object 5 is dust, the cell size of the observed object 5, etc., and determine whether to output the signal as an input signal SS.

[0117] The above-mentioned filters are changed according to the observed object 5. The filters remove noise by smoothing the waveform of the electrical signal ES. Specifically, the filters include a filter that compares the electrical signal ES with a threshold value, a filter that takes a moving average of the electrical signal ES and then compares it with a threshold value, and a filter that differentiates the moving average of the electrical signal ES and then compares it with a threshold value.

[0118] The discrimination unit 101 acquires an input signal SS from the signal input unit 100. The discrimination unit 101 discriminates the observation target 5 observed by the flow cytometer 300 based on the input signal SS acquired from the signal input unit 100.

[0119] The discrimination unit 101 discriminates the input signal SS by a logic circuit, thereby discriminating the observed object 5. By discriminating the observed object 5 by a logic circuit, the discrimination unit 101 can discriminate the observed object 5 faster than a general-purpose computer.

[0120] As described above, the light receiving unit 7 receives scattered light, transmitted light, fluorescent light, or electromagnetic waves from the observation object present in the light irradiation area where the light from the light source is irradiated, and converts them into an electrical signal. The analysis unit 11 analyzes the observation object 5 based on a signal extracted based on the time axis of the electrical signal ES output from the light receiving unit 7. The analysis unit 11 also includes a signal input unit 100. The signal input unit 100 filters the electrical signal ES output by the flow cytometer 300. The signal input unit 100 filters the electrical signal ES to output a noise-reduced signal as an input signal SS to the discrimination unit 101 and the memory unit 9. The machine learning unit 401 can perform machine learning based on the noise-reduced input signal SS, and can improve the accuracy of discrimination of the observed object 5. The signal input unit 100 may be configured by a logic circuit. When the signal input unit 100 is configured by a logic circuit, the filter configuration may be changed based on the machine learning result. Moreover, the analysis unit 11 includes a discrimination unit 101. The discrimination unit 101 includes a logic circuit, and therefore can discriminate the observed object 5 in a shorter time than the calculations required by a general-purpose computer.

[0121] Next, with reference to FIG. 26, a support vector machine method, which is an example of the discrimination algorithm of the above-mentioned analysis unit 11, will be described. FIG. 26 is a diagram showing an example of a discriminant calculation circuit of the analysis unit 11. As shown in FIG.

[0122] The discriminant of the support vector machine method can be expressed by formula (1). The discrimination is performed based on the positive or negative result of formula (1).

[0123]

number

[0124] The b in formula (1) is a constant. Here, by changing the b in formula (1), the discrimination conditions of the support vector machine method can be adjusted. For example, when the b in formula (1) is changed so that the discrimination conditions become stricter, the false positive rate can be kept low. The α and Y in formula (1) are values ​​obtained by machine learning. Here, the X in formula (1) with a ^ (hat symbol) above it is written as X (hat). X (hat) in formula (1) can be expressed by formula (2).

[0125]

number

[0126] X included in formula (2) is a matrix obtained by machine learning. X(hat)jk included in formula (2) is a normalized value of matrix X obtained by machine learning.

[0127]

number

[0128] The x included in formula (3) is data input to the analysis unit 11. In this example, the data input to the analysis unit 11 is a signal extracted based on the time axis of the electrical signal output from the light receiving unit 7. When a ^ (hat symbol) is added above the x included in formula (3), it is written as x (hat). The x (hat)k included in formula (3) is the normalized value of x.

[0129] Here, when the above-mentioned formula (1) is implemented as a logic circuit, the logic circuit scale becomes enormous and may not fit into the logic circuit scale of an FPGA or PLD. Therefore, a logic circuit based on formula (4) is implemented in the logic circuit implemented in the discrimination unit 101.

[0130]

number

[0131] In formula (4), K with a ~ (tilde sign) above it shall be written as K(tilde). In formula (4), X with a ~ (tilde sign) above it shall be written as X(tilde). In formula (4), σ with a ~ (tilde sign) above it shall be written as σ(tilde). βj, K(tilde), X(tilde)jk, and σ(tilde)k included in formula (4) can be expressed by formula (5). Formula (5) is calculated in advance by machine learning unit 401 included in computer 400. The calculated result is incorporated into the logic circuit included in analysis unit 11. b and K(tilde) included in formula (4) are constants, βj and σ(tilde)k are vectors, and X(tilde)jk is a matrix.

[0132]

number

[0133] 26(a) shows the discriminant calculation circuit of the above-mentioned formula (4). The calculation time is shortened by calculating the addition of k included in formula (4) in parallel. By shortening the calculation time, the analysis unit 11 can shorten the time required to discriminate the observed object 5.

[0134] Fig. 26(b) shows a discriminant calculation circuit that calculates the above-mentioned formula (4) at a higher speed. In the discriminant calculation circuit shown in Fig. 26(b), in addition to the above-mentioned configuration of paralleling the addition of k, the addition of j included in formula (4) is calculated in parallel. This allows the analysis unit 11 to discriminate the observed object 5 at a higher speed than the discriminant calculation circuit shown in Fig. 26(a).

[0135] Although a method for implementing the support vector machine technique using a discriminant calculation circuit has been described above, the discrimination algorithm of the discriminator 101 is not limited to this.

[0136] Next, an example will be described in which the analysis device performs machine learning of the observation object 5 observed by flow cytometry.

[0137] 27 is a diagram showing conventional cytometry. Conventional cytometry has a problem in that it is difficult to observe an object at high speed and to change the measurement method for each measurement purpose. FIG. 28 is a diagram showing an example of a processing method for observing an object of observation using the GMI method, which solves the above-mentioned problems. In the GMI method, pattern illumination is applied to an object of observation, such as a cell moving through a flow channel. The object of observation illuminated with pattern illumination emits electromagnetic waves. The electromagnetic waves emitted from the object of observation are detected. Note that the pattern illumination applied to the cells may be illumination that irradiates uniform light. When uniform light is applied to the object of observation, the GMI method transmits the electromagnetic waves emitted from the object of observation through a pattern structure having multiple regions with different electromagnetic wave transmission characteristics. The GMI method detects the electromagnetic waves that have transmitted through the pattern structure. The concept is shown in Figure 29. By directly applying machine learning to time waveform signals, the time required for image reconstruction and feature extraction and analysis from images is shortened, and by analyzing small compressed data, the processing speed is dramatically reduced.

[0138] Figures 30 onwards show a specific implementation example for cell classification. Figure 30 shows three types of cells specifically used. Miapaca and MCf7 have similar sizes and characteristics, while k562 is a size smaller. All are stained green with a dead cell stain (LIVE / DEAD Fixable Green Dead Cell Stain Kit, for 488 nm excitaion, ThermoFisher Scientific) and are classified by machine learning. Only MCF7 is stained blue with nuclear stain (DAPI), which will be used later to verify the accuracy of the classification.

[0139] Figure 30 shows the method for forming a discriminator. Miapaca, MCF7, and k562 are run through the flow path separately to generate time waveform signals generated during GMI imaging. The generated signals are subjected to threshold processing, and each cell type is labeled. These cell type labeled waveform signals are mixed in a computer, and a discriminator is formed to classify them. The Support Vector Machine method was applied as one of the methods for forming a discriminator.

[0140] Next, different cell types (MCF7 and Miapaca in this case) shown in Figure 31 were experimentally mixed, and the time waveform signal generated by GMI was subjected to cell discrimination (classification) using the discriminator prepared previously, and the discrimination results were verified using the total DAPI signal intensity.

[0141] The results are shown in Figure 32. The graph shows the MCF7 concentration as determined by DAPI (blue, correct answer) discrimination when the MCF7 concentration in the mixed solution was changed, as well as the MCF7 concentration as determined by machine learning discrimination of the time waveform of the green fluorescent signal, demonstrating high accuracy (>87%). When comparing MCF7 and Miapaca in green fluorescent images, it is difficult to distinguish with the human eye, clearly demonstrating the usefulness of machine learning. The realization of high-speed, high-precision cell classification has been demonstrated.

[0142] Furthermore, Figures 33 to 35 show examples of fluid system implementation in flow cytometry. Here, the accuracy of the above cell classification is verified when the fluid experimental system is changed, but the correct answer is not determined by the DAPI signal intensity. Instead, a group of time waveforms with cell type labels is mixed in the computer, and this is discriminated (without teaching the correct answer to the machine) to verify the accuracy. As shown in Figure 8, it is generally known that by using microfluidics (flow focusing method), cells can be aligned on the same flow line with a certain width (blur).

[0143] In Figures 34 and 35, the improvement in discrimination accuracy is plotted against the number of time waveforms used for learning when the fluid system is changed. The cell time waveforms used for learning are randomly mixed for each cell. In an ideal fluid experiment system, cells flow along the same streamline and generate a very uniform GMI time waveform. In this case, as shown in Figure 34(a), the discrimination accuracy rises sharply and reaches about 98%. On the other hand, as shown in Figure 34(b), when the flow focus is relaxed and the streamline is given width, the increase in accuracy becomes somewhat more gradual and the achieved accuracy also decreases slightly, but it is still possible to achieve an accuracy of over 95%. However, in actual fluid experiments, there are vibrations in the flow path and instability in the optical system, and a robust discrimination method is required. In this case, by learning with a waveform signal when the flow focus is relaxed and discriminating a waveform signal with a strong flow focus, a robust discrimination accuracy of about 90% can be obtained as shown in Figure 35(a), and the accuracy is stable. On the other hand, as shown in Fig. 35(b), when learning with a waveform signal with strong flow focus and discriminating a waveform signal with loose flow focus, the discrimination accuracy does not reach 90% and the accuracy is unstable. This shows that by learning with a wider range of data, generalization of machine learning can be achieved and practicality can be improved. In addition, in the experiment shown in Fig. 7 above, a mixture of data with strong and loose flow focus was used for learning. In other words, the most robust discrimination accuracy can be obtained by strengthening the flow focus and testing based on the teacher information obtained by learning with a looser flow focus. On the other hand, the most accurate discrimination accuracy can be obtained by strengthening the flow focus and testing based on the teacher information obtained by learning with a stronger flow focus, assuming the conditions are uniform. Furthermore, when testing is based on training information that combines data learned with loosened flow focus and data learned with strengthened flow focus, robust and accurate discrimination accuracy can be obtained.

[0144] In other words, the analysis device performs machine learning of the observation object 5 according to the streamline width adjusted by the streamline width adjustment unit provided in the flow cytometer 300. In the following description, the streamline width is also referred to as the streamline width. The analysis device can perform more accurate discrimination by analyzing the observation object 5 based on training information that combines data obtained by machine learning of the observation object 5 in a state where the streamline width is wider than the diameter of the observation object 5 and data obtained by machine learning of the observation object 5 in a state where the streamline width is according to the diameter of the observation object 5.

[0145] The analytical device of this embodiment includes a flow path through which an object to be observed can move, a light emission unit that emits light to be irradiated onto a light irradiation area of ​​the flow path, a pattern structure unit having a plurality of areas with different light transmission characteristics, a flow path width control unit that variably controls the flow path width through which the object to be observed moving through the flow path can move, a detection unit that detects electromagnetic waves emitted from the object to be observed irradiated with light when the object to be observed in the light irradiation area is irradiated with light, based on the area and the relative movement within the flow path between the light and the object to be observed, an acquisition unit that acquires the change over time in the intensity of the electromagnetic waves detected by the detection unit as an observation result signal that indicates the state of the object to be observed when light is irradiated onto the object to be observed, a teacher information generation unit that generates teacher information by machine learning that indicates a criterion for determining the state of the object to be observed, based on the observation result signal acquired by the acquisition unit and the flow path width at the time the observation result signal was acquired, and an estimation unit that estimates the state of the object to be observed moving through the flow path, based on the observation result signal acquired by the acquisition unit and the teacher information generated by the teacher information generation unit.

[0146] In addition, the analytical device may have a flow path width control unit that controls the flow path width to a first flow path width that is a width corresponding to the diameter of the object to be observed, a teacher information generation unit that generates first teacher information as teacher information based on a first observation result signal detected by the detection unit at the first flow path width controlled by the flow path width control unit, and an estimation unit that estimates the state of the object to be observed moving through the flow path based on the first teacher information generated by the teacher information generation unit and the observation result signal acquired by the acquisition unit.

[0147] In addition, the analytical device may have a flow path width control unit that controls the flow path width to a second flow path width that is based on the diameter of the object to be observed and is wider than the first flow path width, and the teacher information generation unit further generates second teacher information as teacher information based on a second observation result signal detected by the detection unit at the second flow path width controlled by the flow path width control unit, and the estimation unit may estimate the state of the object to be observed moving through the flow path based on the first teacher information generated by the teacher information generation unit, the second teacher information generated by the teacher information generation unit, and the observation result signal acquired by the acquisition unit.

[0148] In addition, the analytical device may have a flow path width control unit that controls the flow path width to a first flow path width that is based on the diameter of the object to be observed and is narrower than the second flow path width, and the teacher information generation unit further generates first teacher information as teacher information based on a first observation result signal detected by the detection unit at the first flow path width controlled by the flow path width control unit, and the estimation unit may estimate the state of the object to be observed moving through the flow path based on the first teacher information generated by the teacher information generation unit, the second teacher information generated by the teacher information generation unit, and the observation result signal acquired by the acquisition unit. [Explanation of symbols]

[0149] 1 light source 3 Light irradiation area 5. Observation Object 7 Light receiving section 9 Storage section 11 Analysis Department 13 Optical system control section 21 Multiple light regions 25 Light receiving area 31 Flow Cell 33 Sorting Department 200 Analysis System 300 Flow Cytometer 400 Computers 401 Machine Learning Department

Claims

1. On the computer, an input step of inputting optical signal information acquired from an object to be observed by a ghost cytometry method; a discrimination step of discriminating the observed object from the optical signal information; an output step for outputting the determined result; A program for executing the above.

2. The object to be observed is any one of a cell, a body fluid, and an eyeball. The program according to claim 1.

3. The optical signal information is acquired for the observation object moving through the flow path. The program according to claim 1.

4. The optical signal information indicates a time waveform signal of the optical intensity of the light emitted from the object to be observed. The program according to claim 1.

5. The optical signal information includes spatial information capable of reconstructing a two-dimensional image of the observed object, compressed as one-dimensional data. The program according to claim 1.

6. The discrimination step discriminates the observed object from the optical signal information based on a trained model. The program according to claim 1.

7. The trained model is created by machine learning that learns characteristics of the observed object from the optical signal information. The program according to claim 6.

8. The learning for creating the trained model is performed by at least one of supervised machine learning and unsupervised machine learning, or a combination thereof. The program according to claim 6.

9. The determining step determines whether the observed object is a specific object. The program according to claim 1.

10. The computer is a computer including a logic circuit configured by at least one of a programmable logic device and an ASIC (Application Specific Integrated Circuit), or a combination thereof. The program according to claim 1.

11. The ghost cytometry method is a method in which an object to be observed is irradiated with illumination light having an optical pattern and the light emitted from the object to be observed is detected by a photodetector. The program according to claim 1.

12. The ghost cytometry method is a method in which an optical pattern is imparted to light emitted from an object to be observed by irradiating the object with illumination light, and the light is detected by a photodetector. The program according to claim 1.

13. The object to be observed is detected by the photodetector while moving relative to the optical pattern. The program according to claim 11 or 12.

14. A storage device for recording the program according to claim 1.

15. An analysis device that discriminates the observed object by causing a computer to execute the program according to claim 1.

Images