Image analysis device, image analysis method, and program, and compound-eye optical system quantum measurement system, and compound-eye optical system quantum measurement method
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- HIROSHIMA UNIVERSITY
- Filing Date
- 2023-08-07
- Publication Date
- 2026-07-03
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an image analysis device, an image analysis method and program, as well as a compound eye optical system quantum measurement system and a compound eye optical system quantum measurement method, and in particular to a quantum measurement technology that analyzes multi-view images of a sample containing a quantum sensor that fluoresces under specific conditions to measure physical quantities within the sample. [Background technology]
[0002] To understand phenomena such as disease, cell aging, cancer, and cell responses to radiation, we need sensors that respond sensitively to minute changes in cells and molecules. For example, protein-type, polymer-type, and fluorescent small molecule-type temperature measurement probes have been developed to measure intracellular temperature. Such bioimaging using fluorescence has become an essential technique for analyzing the localization and function of molecules in cells and living organisms.
[0003] To accurately understand very small changes in living organisms caused by life phenomena, ultra-sensitive sensors that respond sensitively to slight changes in cells and molecules are required. One such sensor is a quantum sensor that uses quantum effects to measure physical quantities such as magnetism, magnetic field, and temperature. Among them, fluorescent nanodiamonds (FNDs) have attracted attention as highly sensitive fluorescent labeling agents in cell imaging and trace virus detection. In order to measure molecules in cells using a quantum sensor, the size of the quantum sensor needs to be several nm, which is the molecular size, because a size of several tens of nm is too large. The inventors of the present invention have succeeded in creating the world's smallest FNDs of several nm in size, and have overcome the drawback that background light such as autofluorescence and fluorescence of impurities interferes with the fluorescence detection of quantum sensors in fluorescence imaging using FNDs, thereby improving the signal-to-background ratio (SBR value) (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]
[0004] [Non-Patent Document 1] Tamami Yanagi, Kiichi Kaminaga, Michiyo Suzuki, Hiroshi Abe, Hiroki Yamamoto, Takeshi Ohshima, Akihiro Kuwahata, Masaki Sekino, Tatsuhiko Imaoka, Shizuko Kakinuma, Takuma Sugi, Wataru Kada, Osamu Hanaizumi, and Ryuji Igarashi, “All-Optical Wide-Field Selective Imaging of Fluorescent Nanodiamonds in Cells, In Vivo and Ex Vivo,” ACS Nano 2021 15 (8), 12869-12879, August 2, 2021 Summary of the Invention [Problem to be solved by the invention]
[0005] In quantum sensing using a quantum sensor, errors or variations in the fluorescence intensity of the quantum sensor due to the performance of the optical system or imaging system, etc., will also cause errors in the measured physical quantity. Measurement errors or variations can be reduced by repeating measurements and taking the average, but this takes a long time to measure.
[0006] Therefore, an object of the present invention is to make it possible to measure a physical quantity with high accuracy in quantum sensing with fewer measurements, preferably with one measurement, without repeating the measurements many times. [Means for solving the problem]
[0007] The image analysis device according to aspect 1 of the present invention is an image analysis device that analyzes a multi-viewpoint image of a sample containing a quantum sensor, captured by a compound eye optical system while the sample is irradiated with excitation light of a wavelength that excites the quantum sensor to fluoresce and microwaves of a frequency that causes the quantum sensor to undergo optical detection magnetic resonance, and is equipped with a fluorescence intensity determination unit that determines the fluorescence intensity of the quantum sensor from the fluorescence light of the quantum sensor captured in multiple element images in the multi-viewpoint image of the sample, and a physical quantity calculation unit that calculates a physical quantity indicated by the quantum sensor from the fluorescence intensity of the quantum sensor.
[0008] An image analysis device according to a second aspect of the present invention is the image analysis device according to the first aspect, in which the multi-viewpoint images of the sample are obtained by continuously imaging the sample by sweeping the microwave frequency around the resonance frequency of the photodetection magnetic resonance of the quantum sensor, and the physical quantity calculation unit identifies the resonance frequency of the photodetection magnetic resonance of the quantum sensor from a change pattern of the fluorescence intensity of the quantum sensor, and calculates the physical quantity corresponding to the identified resonance frequency.
[0009] An image analysis device according to a third aspect of the present invention is an image analysis device according to the second aspect, in which the multi-view image of the sample is obtained by continuously imaging the sample while further turning the microwave on and off at a constant cycle, and the fluorescence intensity determination unit determines, as the fluorescence intensity of the quantum sensor, the relative value of the fluorescence intensity of the quantum sensor captured in the multi-view image when the microwave is on and off, which are successive in time series.
[0010] An image analysis device according to a fourth aspect of the present invention is the image analysis device according to the second aspect, wherein the multi-view image of the sample is obtained by continuously imaging the sample while further turning the microwave on and off at a constant cycle, and the image analysis device further includes a fluorescent light beam identification unit that identifies light beams that become bright and dark in the multi-view image with the on / off cycle of the microwave as fluorescent light beams of the quantum sensor, and the fluorescence intensity identification unit determines the intensity of the identified fluorescent light beams.
[0011] The image analysis device according to a fifth aspect of the present invention is the image analysis device according to the first aspect, wherein the fluorescence intensity identification unit selects, from among a plurality of element images capturing the fluorescent light beam of the quantum sensor, an element image in which the fluorescent light beam is stronger than a certain level, and calculates the fluorescence intensity of the quantum sensor by averaging the pixel values of the fluorescent light beam of the quantum sensor captured in the selected element image.
[0012] An image analysis device according to a sixth aspect of the present invention is an image analysis device according to any one of the first to fifth aspects, wherein the quantum sensor is a fluorescent nanodiamond.
[0013] An image analyzing device according to a seventh aspect of the present invention is the image analyzing device according to any one of the first to sixth aspects, wherein the physical quantity is temperature.
[0014] An image analysis method according to aspect 8 of the present invention is an image analysis method for analyzing a multi-viewpoint image of a sample containing a quantum sensor captured by a compound eye optical system while the sample is irradiated with excitation light of a wavelength that excites the quantum sensor to fluoresce and microwaves of a frequency that causes the quantum sensor to undergo optical detection magnetic resonance, and includes a step in which a fluorescence intensity identification unit determines the fluorescence intensity of the quantum sensor from the fluorescence light of the quantum sensor captured in multiple element images in the multi-viewpoint image of the sample, and a step in which a physical quantity calculation unit calculates the physical quantity indicated by each quantum sensor from the fluorescence intensity of the quantum sensor.
[0015] A program according to a ninth aspect of the present invention is a program that causes a computer to analyze a multi-viewpoint image of a sample containing a quantum sensor, captured by a compound eye optical system, while the sample is irradiated with excitation light of a wavelength that excites the quantum sensor to fluoresce and microwaves of a frequency that causes the quantum sensor to undergo optical detection magnetic resonance, and causes the computer to function as a fluorescence intensity determination means that determines the fluorescence intensity of the quantum sensor from the fluorescent light of the quantum sensor captured in multiple element images in the multi-viewpoint image of the sample, and a physical quantity calculation means that calculates the physical quantity indicated by the quantum sensor from the fluorescence intensity of the quantum sensor.
[0016] The compound eye optical system quantum measurement system of aspect 10 of the present invention is a system for measuring a physical quantity using a quantum sensor, and comprises an excitation light generator that generates excitation light of a wavelength that excites the quantum sensor to fluoresce, a microwave generator that generates microwaves of a frequency that causes the quantum sensor to undergo optical detection magnetic resonance, a compound eye optical system that captures a multi-view image of a subject, and an image analysis device of any one of aspects 1 to 7 that analyzes the multi-view image of the sample captured by the compound eye optical system while the excitation light and the microwaves are irradiated onto a sample including the quantum sensor.
[0017] The compound eye optical system quantum measurement method of aspect 11 of the present invention is a method for measuring a physical quantity using a quantum sensor, and includes a step in which the compound eye optical system captures a multi-viewpoint image of a sample containing the quantum sensor while irradiating the sample with excitation light of a wavelength that excites the quantum sensor to fluoresce and microwaves of a frequency that causes the quantum sensor to optically detect magnetic resonance, and a step in which an image analysis device of any one of aspects 1 to 7 analyzes the multi-viewpoint image of the sample. Effect of the Invention
[0018] According to the present invention, by analyzing multi-view images of a sample including a quantum sensor captured from multiple different viewpoints, it is possible to measure a physical quantity with high accuracy with fewer measurements, preferably with a single measurement, without having to repeat the measurements multiple times. [Brief description of the drawings]
[0019] [Figure 1] 1 is a schematic diagram of a compound eye optical system quantum measurement system according to an embodiment of the present invention. [Diagram 2] 2 is a schematic diagram of a compound eye optical system, a storage device, and an image analysis device in the compound eye optical system quantum measurement system of FIG. 1. [Diagram 3] 1 is a flowchart showing an example of a method for acquiring a multi-viewpoint image of a sample. [Figure 4] 1 is a flowchart showing an example of a method for analyzing a multi-viewpoint image of a sample. [Diagram 5]1A and 1B are diagrams comparing a multi-viewpoint image of a sample captured with a compound eye optical system and a wide-field image of a sample captured with a monocular optical system. [Figure 6] FIG. 1 is a schematic diagram showing the difference in intensity fluctuation between quantum sensor fluorescence captured in a multi-view image of a sample and other light. [Figure 7] 11 is a graph illustrating the relationship between the number of regions of interest used in averaging and the measured temperature error. [Figure 8] 1 is a graph illustrating the temperature dependence of the ODMR spectrum of a fluorescent nanodiamond, which is an example of a quantum sensor. [Figure 9] 11 is a graph illustrating an example of the relationship between the number of integrations and the measurement temperature error in temperature measurements based on a wide-field image and a multi-viewpoint image. [Figure 10] 11 is a graph illustrating an example of the relationship between excitation light intensity and measurement temperature error in temperature measurements based on wide-field images and multi-viewpoint images. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0020] Hereinafter, the embodiments of the present invention will be described in detail with reference to the drawings as appropriate. However, more detailed explanations than necessary may be omitted. For example, detailed explanations of already well-known matters or duplicate explanations of substantially identical configurations may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the inventor provides the accompanying drawings and the following explanation so that those skilled in the art can fully understand the present invention, and does not intend to limit the subject matter described in the claims by them. In addition, the dimensions of each member depicted in the drawings, detailed shapes of details, etc. may differ from the actual ones.
[0021] <Embodiment> 1 is a schematic diagram of a compound-eye optical system quantum measurement system according to an embodiment of the present invention. The compound-eye optical system quantum measurement system 100 according to this embodiment captures a multi-viewpoint image of a sample including a quantum sensor from a plurality of different viewpoints, analyzes the image, and measures a physical quantity in the sample. Specifically, the compound-eye optical system quantum measurement system 100 includes, as main components, a compound-eye optical system 10, a storage device 20, an image analysis device 30, an excitation light generator 40, a microwave generator 50, and a synchronization signal generator 60.
[0022] For example, fluorescent nanodiamonds (FNDs) can be used as quantum sensors. FNDs are artificial diamond particles with diameters ranging from a few nm to a few hundred nm, and one of their characteristics is that they contain nitrogen atoms (Nitrogen) as impurities in the crystal. This nitrogen atom and the vacancy (vacancy) next to it form an NVC (Nitrogen-Vacancy Center), which emits a photon of 637 nm when excited with green light of 532 nm (red fluorescence). The fluorescence of FNDs is highly light-resistant and does not fade or blink, making it suitable for long-term image measurement. Furthermore, NVCs contain negatively charged NVs consisting of a total of six electrons: two from the nitrogen atom, three from the carbon atom, and one in the vacancy. - There exists a cavity in which the electron spin resonates with electromagnetic waves, causing a change in the fluorescence intensity of the FND, an phenomenon known as optically detected magnetic resonance (ODMR). Specifically, when microwaves of around 2.87 GHz are irradiated onto an FND, the fluorescence intensity of the FND weakens due to ODMR activity. Since this resonance frequency is temperature-dependent, it is possible to measure temperature as a physical quantity by measuring the change in the resonance frequency of the optically detected magnetic resonance of the FND. For example, it is possible to measure the temperature of a microscopic region within the biological tissue of the nematode C. elegans by measuring the fluorescence of an FND incorporated into the nematode C. elegans as a sample.
[0023] The excitation light generator 40 is a device that generates excitation light of a wavelength that excites the quantum sensor to fluoresce. When the FND is used as the quantum sensor, the excitation light generator 40 outputs a green laser light of 532 nm.
[0024] The microwave generator 50 is a device that generates microwaves at a frequency that causes the quantum sensor to optically detect magnetic resonance. When the FND is used as a quantum sensor, the microwave generator 50 generates microwaves at about 2.87 GHz, which is the resonance frequency of the electron spin of the NVC in the FND, for example, 2.82 GHz to 2.92 GHz. The microwave generator 50 can also sweep the frequency to generate microwaves, and can output the microwaves in a pulsed form in synchronization with the synchronization signal output from the synchronization signal generator 60.
[0025] The sample is adjusted to a sample holder such as a slide glass or a petri dish and placed on the stage. An objective lens is placed below the stage, and laser light output from an excitation light generator 40 enters the objective lens via a collimator lens, a condenser lens, a dichroic mirror, etc., and the excitation light is irradiated from the tip of the objective lens onto the sample in the sample holder. Meanwhile, a coil is placed directly above the sample holder placed on the stage, and microwaves output from a microwave generator 50 are transmitted through a coaxial cable and irradiated from the coil at the tip onto the sample in the sample holder.
[0026] The synchronization signal generator 60 is a device that generates a synchronization signal for controlling the frequency sweep of the microwave generator 50, the on / off control of the microwave generation, and the imaging timing of the compound eye optical system 10. Specifically, the synchronization signal generator 60 is a function generator.
[0027] The compound eye optical system 10, the storage device 20, and the image analysis device 30 will be described with reference to another drawing. FIG. 2 is a schematic diagram of the compound eye optical system 10, the storage device 20, and the image analysis device 30 in the compound eye optical system quantum measurement device 100 of FIG. 1. The compound eye optical system 10 is an optical system that captures an image of the same subject from multiple viewpoints that are shifted from each other, and includes a main lens 1, a microlens array 2, and an image sensor 3. A light field optical system is an example of the compound eye optical system 10. The main lens 1 may include an objective lens, an imaging lens, an absorption filter, and the like, but for convenience, the main lens 1 will be referred to as including them. The microlens array 2 is configured by arranging multiple microlenses (lenslets) 2a two-dimensionally, and is disposed between the main lens 1 and the image sensor 3. The image sensor 3 is a CCD sensor or a CMOS sensor configured by arranging multiple light receiving elements (not shown) two-dimensionally, which photoelectrically convert input light and output an electrical signal. The electrical signals output from each light receiving element are converted into digital data by an A / D converter (not shown), and a multi-viewpoint image of the same subject captured from multiple mutually shifted viewpoints is output from the compound eye optical system 10. A light field image is an example of a multi-viewpoint image.
[0028] The storage device 20 is a collection of storage devices such as RAM, ROM, SSD, and HDD. The RAM is mainly used as a working memory when the image analysis device 30 performs image analysis. The ROM, SSD, and HDD mainly store computer programs for operating the image analysis device 30, multi-viewpoint images of a sample that is the analysis target of the image analysis device 30, and the like, temporarily or for a long period of time. The storage device 20 may be located on a cloud.
[0029] The image analysis device 30 analyzes a multi-view image of a sample stored in the storage device 20 and calculates the physical quantity indicated by the quantum sensor in the image. Specifically, the image analysis device 30 includes a fluorescent light beam discrimination unit 31, a fluorescent intensity identification unit 32, and a physical quantity calculation unit 33. Each of these components can be realized as hardware such as an ASIC or FPGA, or as software that executes a computer program stored in the storage device 20 or a computer program stored and provided in a computer-readable non-transitory recording medium on a processor (not shown) such as a CPU or GPU. The image analysis device 30 can also be realized by appropriately combining hardware and software. During image analysis, the image analysis device 30 appropriately accesses various storage devices constituting the storage device 20 to write and read data. All or part of the components of the image analysis device 30 may be located on a cloud. The analysis results by the image analysis device 30 are passed to another processing system for further processing, or the results are displayed to the user through a user interface (not shown). The storage device 20 and the image analysis device 30 may be configured as an integrated computer device.
[0030] Next, a method for acquiring a multi-view image of a sample to be analyzed by the image analysis device 30 will be described. FIG. 3 is a flow chart showing an example of a method for acquiring a multi-view image of a sample. First, a sample holder containing a sample including a quantum sensor is placed on a stage, and the sample is irradiated with excitation light output from the excitation light generator 40 (S1). Next, the oscillation frequency of the microwave generator 50 is set to a start value near the resonance frequency of the photodetection magnetic resonance of the quantum sensor, and a frequency sweep is started (S2). In synchronization with the synchronization signal of the synchronization signal generator 60, the microwave generator 50 turns on the microwave, and the compound eye optical system 10 captures a multi-view image of the sample in a state where the excitation light and microwave are irradiated, and stores the image in the storage device 20 (S3). Next, in synchronization with the synchronization signal of the synchronization signal generator 60, the microwave generator 50 turns off the microwave, and the compound eye optical system 10 captures a multi-view image of the sample in a state where the microwave is not irradiated but only the excitation light is irradiated, and stores the image in the storage device 20 (S4). Thereafter, the microwave generator 50 sweeps the frequency (S5), and steps S3 and S4 are repeated until the microwave frequency reaches a sweep end value near the resonant frequency of the optically detected magnetic resonance of the quantum sensor.
[0031] For example, when the microwave frequency is swept over 100 points in synchronization with a 400 Hz synchronization signal, 200 multi-view images are acquired in 0.5 seconds, alternating between multi-view images of the sample irradiated with excitation light and microwaves and multi-view images of the sample irradiated with excitation light only.
[0032] The microwave may be swept so that the frequency increases or decreases monotonically, and the frequency sweep step does not have to be a constant value. Frequency hopping may be used instead of frequency sweep. It is sufficient that multiple multi-viewpoint images of the sample can be obtained when microwaves of multiple different frequencies are irradiated within a certain frequency range.
[0033] In addition, depending on the performance and frame rate of the compound eye optical system 10, the fluorescent light of the quantum sensor captured in one multi-view image may be extremely weak. In such a case, the pixel values of the fluorescent light of the quantum sensor may be integrated to make the light intensity a sufficiently large value, and then various analysis processes described below may be performed. Therefore, the above steps S2 to S5 may be repeated multiple times to obtain more sets of multi-view images so that the light intensity can be integrated in the image analysis process.
[0034] Next, a method for analyzing the acquired multi-viewpoint images of a sample by the image analysis device 30 will be described. Fig. 4 is a flow chart showing an example of a method for analyzing the multi-viewpoint images of a sample. First, the image analysis device 30 reads the multi-viewpoint images of the sample that have been continuously captured from the storage device 20 (S11).
[0035] Figure 5 compares a multi-view image of a sample captured with a compound eye optical system and a wide-field image of a sample captured with a monocular optical system. Both images are of a sample containing one quantum sensor. The rectangular frame in the figure represents the region of interest (ROI). The ROI here is the area that captures the fluorescent light from the quantum sensor. In both the multi-view image and the wide-field image, the fluorescence intensity of the quantum sensor can be calculated from the pixels in the region of interest.
[0036] In a monocular optical system, the subject is captured from a single viewpoint, so the fluorescence of the quantum sensor is recorded as a single light image in the white-field image. On the other hand, in a compound-eye optical system, the same subject is captured from multiple different viewpoints, so the fluorescence of the quantum sensor is recorded as multiple light images in the multi-viewpoint image. In other words, a multi-viewpoint image is made up of multiple elemental images of the same subject captured from multiple viewpoints, and the fluorescence of the quantum sensor is captured in each of the multiple elemental images in the multi-viewpoint image.
[0037] Returning to FIG. 4, the fluorescent light beam identifying unit 31 identifies the fluorescent light beam of the quantum sensor in the read multi-view image (S12). FIG. 6 is a schematic diagram showing the difference between the quantum sensor fluorescence captured in the multi-view image of the sample and the intensity fluctuation of other light. As described above, when the quantum sensor is irradiated with a microwave having a frequency near the resonant frequency of the optical detection magnetic resonance, the fluorescence intensity becomes weak. Therefore, in the multi-view image obtained by continuously imaging the sample while turning on and off the microwave in synchronization with the constant periodic synchronization signal output from the synchronization signal generator 60, the fluorescence intensity of the quantum sensor changes microscopically according to the on / off period of the microwave, while macroscopically according to the frequency sweep of the microwave. On the other hand, the intensity of background light such as the autofluorescence emitted by the sample itself and the fluorescence of impurities changes regardless of the on / off period of the microwave. Due to the difference in the characteristics of such intensity changes, the fluorescence of the quantum sensor can be distinguished from other light in the multi-view image of the sample continuously imaged. Specifically, the fluorescent light beam identifying unit 31 identifies the light beam that becomes bright and dark with the on / off cycle of the microwave in the multi-view image of the sample as the fluorescent light beam of the quantum sensor. This makes it possible to efficiently extract the fluorescence of the quantum sensor from the multi-view image that includes background light, and improve the SBR value (signal / background light ratio).
[0038] 4, in steps S13 and S14, the fluorescence intensity identifying unit 32 determines the fluorescence intensity of the quantum sensor from the fluorescence light beam of the quantum sensor captured in multiple elemental images in the multi-viewpoint image of the sample. Since the fluorescence light beam of the quantum sensor has been identified by the fluorescence light beam identifying unit 31 in step S12, the fluorescence intensity identifying unit 32 need only determine the intensity of the light beam identified as the fluorescence light beam of the quantum sensor.
[0039] More specifically, the fluorescence intensity identifying unit 32 averages the pixel values of the pixels in the region of interest set in each element image, and obtains the average value as the fluorescence intensity of the quantum sensor. The region of interest may be the same size as the element image, or it may be a region smaller than the element image.
[0040] It is not necessary to subject all elemental images in which the quantum sensor's fluorescent light was captured to this averaging. A multi-viewpoint image of a sample contains elemental images in which the quantum sensor's fluorescent light was captured from various viewpoints, but among these are elemental images with very weak light that is difficult to distinguish from the background. If the pixel values of the pixels of such elemental images are included in the averaging, this will cause the S / N ratio of the average value to deteriorate.
[0041] Figure 7 is a graph illustrating the relationship between the number of regions of interest used in averaging and the measurement temperature error. As shown in the graph, the measurement temperature error is smaller when there is more than one region of interest. However, once there is a certain number of regions of interest, the measurement temperature error does not become smaller. Therefore, it is not necessarily true that the more regions of interest, the better, and it is acceptable to limit the number to a certain number, for example, to those where the fluorescent light of the quantum sensor is stronger than a certain level.
[0042] 4, the fluorescence intensity specifying unit 32 selects, from among a plurality of elemental images capturing the fluorescent light of the quantum sensor, an elemental image in which the fluorescent light is stronger than a certain level (S13).Then, the fluorescence intensity specifying unit 32 averages the pixel values of the fluorescent light of the quantum sensor captured in the selected elemental image to obtain the fluorescence intensity of the quantum sensor (S14).
[0043] The fluorescence intensity determination unit 32 may determine the relative value of the fluorescence intensity of the quantum sensor captured in the multi-viewpoint image when the microwave is on and off in chronological order as the fluorescence intensity of the quantum sensor. The relative value may be the ratio or difference of the fluorescence intensity. In the example of FIG. 6, the relative value of the intensity that rises and falls microscopically may be regarded as the fluorescence intensity of the quantum sensor. This makes it possible to more accurately determine the fluorescence intensity of the quantum sensor by eliminating the influence of background light and the depth position of the quantum sensor on the fluorescence intensity.
[0044] Furthermore, the fluorescence intensity specifying unit 32 obtains the fluorescence intensity of each quantum sensor for each microwave frequency point. This is to know the change pattern of how the fluorescence intensity of the quantum sensor changes due to the microwave frequency sweep. The fluorescence intensity specifying unit 32 may integrate the pixel values of the fluorescent light beam of the quantum sensor for each frequency point of the swept microwave so that the analysis process can be performed with a sufficiently large light beam intensity. This makes it possible to know the change pattern of the fluorescence intensity with high accuracy.
[0045] When the fluorescence intensity of the quantum sensor is specified, the physical quantity calculation unit 33 specifies the resonance frequency of the optical detection magnetic resonance of the quantum sensor from the change pattern of the fluorescence intensity of the quantum sensor, and calculates the physical quantity corresponding to the specified resonance frequency (S15). FIG. 8 is a graph illustrating the temperature dependence of the ODMR spectrum of a fluorescent nanodiamond (FND), which is an example of a quantum sensor. The horizontal axis of the graph is the frequency of the microwave irradiated to the sample, and the vertical axis is the relative value of the fluorescence intensity of the FND specified by the fluorescence intensity specification unit 32. In the graph, the relative values of the fluorescence intensity of the FND at each frequency point under two different environments (when the ambient temperatures of the quantum sensor are 24° C. and 28° C.) are plotted, and the respective fitting curves are superimposed. Each fitting curve was obtained by fitting the fluorescence intensity of the FND at each frequency point to a bimodal Lorentzian function.
[0046] As can be seen from the graph, the fitting curve of the ODMR spectrum of the FND has the characteristic that as the irradiated microwave approaches the resonance frequency, it decreases significantly and takes a minimum value, and then increases slightly near the resonance frequency and takes a maximum value at the resonance frequency. Furthermore, the resonance frequency of the FND varies depending on the temperature. That is, the resonance frequency is temperature dependent. Therefore, if the resonance frequency of the FND can be determined, the temperature as a physical quantity can be determined from it. The temperature can be measured from the resonance frequency of the FND with an accuracy of about 0.1 to 1°C.
[0047] Specifically, the physical quantity calculation unit 33 can determine the resonance frequency of the quantum sensor from the fitting curve by fitting the fluorescence intensity at each microwave frequency point identified by the fluorescence intensity identification unit 32 to a function. For the fitting, a bimodal Lorentzian function or a Gaussian function can be used. Alternatively, the physical quantity calculation unit 33 may use a machine learning model that has been trained in advance with the fluorescence intensity at each microwave frequency point as an input value and the resonance frequency as an output value, and input the fluorescence intensity of the quantum sensor at each microwave frequency point identified by the fluorescence intensity identification unit 32 into the machine learning model to predict the resonance frequency.
[0048] In addition, the corresponding temperature varies even for the same resonance frequency due to variations in the size of the quantum sensor, etc. Therefore, it is preferable to obtain the resonance frequency of the quantum sensor in advance under a predetermined temperature environment, specify the correspondence between the temperature and the resonance frequency, and use this as a reference to calculate the temperature for the resonance frequency specified from the strength of the quantum sensor contained in the sample. Effect
[0049] FIG. 9 is a graph illustrating the relationship between the number of integrations and the measurement temperature error in temperature measurement based on a wide-field image and a multi-view image. The horizontal axis of the graph is the number of integrations, which is the number of frequency sweeps of microwaves, and the vertical axis is the temperature error when temperature measurement is attempted six times for one quantum sensor. The error bars for each number of integrations represent the variation in the error of the temperature measured in each trial when temperature measurement is attempted for each of five quantum sensors prepared as samples. The measurement temperature error is reduced by integrating multiple times for both the wide-field image and the multi-view image, that is, by repeating the frequency sweep and averaging. It is worth noting that the measurement error of the temperature measurement based on the multi-view image is smaller in one measurement than that of the temperature measurement based on the wide-field image. In addition, the temperature measurement based on the multi-view image can achieve a measurement error equivalent to the five-time integration of the temperature measurement based on the wide-field image with two integrations. Therefore, according to the compound-eye optical system quantum measurement system 100 according to this embodiment, it is possible to measure a physical quantity with high accuracy in one measurement without repeating the measurement many times.
[0050] FIG. 10 is a graph illustrating the relationship between excitation light intensity and measured temperature error in temperature measurements based on wide-field images and multi-viewpoint images. The horizontal axis of the graph is the intensity of the excitation light irradiated to the quantum sensor, and the vertical axis is the temperature error when temperature measurements are performed six times for one quantum sensor. The error bars for each excitation light intensity represent the variation in the temperature error measured in each trial when temperature measurements are performed six times for each of the five quantum sensors prepared as samples. In temperature measurements based on wide-field images, there is a tendency for the measured temperature error to increase as the excitation light intensity decreases, but in temperature measurements based on multi-viewpoint images, the measured temperature error is generally kept small regardless of the intensity of the excitation light. In addition, in the illustrated graph, the measured temperature error is minimized when the excitation light intensity is 100 mW in temperature measurements based on wide-field images, and the measured temperature error is minimized when the excitation light intensity is 90 mW in temperature measurements based on multi-viewpoint images, but there is a difference of about 2.5 times between the two, and the temperature measurement based on multi-viewpoint images has overwhelmingly better measurement accuracy. Therefore, according to the compound eye optical system quantum measurement system 100 of this embodiment, even if the fluorescence of the quantum sensor is weak, it is possible to measure the physical quantity with high accuracy, and it is possible to measure the physical quantity with significantly higher accuracy than physical quantity measurement based on a wide-field image.
[0051] <<Variations>> If there is no need to distinguish between the fluorescence of the quantum sensor and background light in a multi-viewpoint image of a sample, the fluorescent light beam discriminating unit 31 and step S12 may be omitted.
[0052] Image analysis may be performed by treating the fluorescence intensity when the microwave is on as the fluorescence intensity of the quantum sensor without calculating the relative value of the fluorescence intensity of the quantum sensor when the microwave is on and when the microwave is off. In this case, since it is not necessary to turn the microwave of the microwave generator 50 on and off, step S4 can be omitted.
[0053] When measuring temperature using fluorescent nanodiamonds as quantum sensors, it is necessary to sweep the microwave frequency to obtain the change pattern of the fluorescent intensity of the fluorescent nanodiamonds and then identify the resonance frequency, but when using a quantum sensor whose physical quantity is uniquely determined from the fluorescent intensity when irradiated with a microwave of a predetermined frequency, it is not necessary to sweep the microwave. In this case, steps S2 and S5 can be omitted. Furthermore, if there is no need to distinguish between the fluorescence of the quantum sensor and background light in the multi-view image of the sample, the synchronization signal generator 60 can be omitted.
[0054] Either the relative value of the fluorescence intensity of the quantum sensor captured in the multi-viewpoint image when the microwave is on and off in a time series successively, or the averaging of the pixel values of the pixels in the region of interest set in each element image may be performed first. That is, the relative value of the fluorescence intensity of the quantum sensor may be obtained first and then averaged for multiple element images, or the average value of the pixel values of multiple element pixels may be obtained for each of the times when the microwave is on and off in a time series successively, and then the ratio or difference of the average values may be obtained.
[0055] Although an example of measuring temperature using fluorescent nanodiamonds as a quantum sensor has been explained, it is also possible to measure physical quantities other than temperature, such as electric fields and magnetic fields, by applying an external magnetic field to a sample using the same principle. For example, the following document discloses a method of measuring weak magnetic fields using fluorescent nanodiamonds as a magnetic sensor. Reference 1: Balasubramanian, G., Chan, I., Kolesov, R. et al. Nanoscale imaging magnetometry with diamond spins under ambient conditions. Nature 455, 648-651 (2008). https: / / doi.org / 10.1038 / nature07278 Reference 2: Maze, J., Stanwix, P., Hodges, J. et al. Nanoscale magnetic sensing with an individual electronic spin in diamond. Nature 455, 644-647 (2008). https: / / doi.org / 10.1038 / nature07279 Reference 3: Taylor, J., Cappellaro, P., Childress, L. et al. High-sensitivity diamond magnetometer with nanoscale resolution. Nature Phys 4, 810-816 (2008). https: / / doi.org / 10.1038 / nphys1075
[0056] As described above, the embodiment has been described as an example of the technology of the present invention. For this purpose, the attached drawings and detailed description have been provided. Therefore, among the components described in the attached drawings and detailed description, not only components essential for solving the problem but also components that are not essential for solving the problem in order to illustrate the above technology may be included. Therefore, just because those non-essential components are described in the attached drawings or detailed description, it should not be immediately recognized that those non-essential components are essential. In addition, since the above-mentioned embodiment is intended to illustrate the technology of the present invention, various modifications, substitutions, additions, omissions, etc. may be made within the scope of the claims or their equivalents. [Industrial Applicability]
[0057] INDUSTRIAL APPLICABILITY The compound eye optical system quantum measurement system and image analysis device according to the present invention are capable of measuring physical quantities within a sample with high accuracy, and are therefore useful for observing phenomena such as biomolecules and intracellular behavior. [Explanation of symbols]
[0058] 100 Compound Eye Optical System Quantum Measurement System 10 Compound eye optical system 30 Image analysis device 31 Fluorescent light identification unit 32 Fluorescence Intensity Identification Unit 33 Physical quantity calculation section 40 Excitation Light Generator 50 Microwave Generator 60 Synchronous signal generator
Claims
1. An image analysis device that analyzes a multi-viewpoint image of a sample including a quantum sensor, the sample being imaged by a compound eye optical system in a state where the sample is irradiated with excitation light having a wavelength that excites the quantum sensor with fluorescence and microwaves having a frequency that causes the quantum sensor to optically detect magnetic resonance, a fluorescence intensity determination unit that determines the fluorescence intensity of the quantum sensor from the fluorescence light of the quantum sensor captured in a plurality of element images in the multi-viewpoint image of the sample; a physical quantity calculation unit that calculates a physical quantity indicated by the quantum sensor from the fluorescence intensity of the quantum sensor; An image analysis device comprising:
2. The multi-viewpoint images of the sample are obtained by continuously imaging the sample by sweeping the frequency of the microwave around the resonant frequency of the optically detected magnetic resonance of the quantum sensor; The physical quantity calculation unit identifies a resonance frequency of the optical detection magnetic resonance of the quantum sensor from a change pattern of the fluorescence intensity of the quantum sensor, and calculates the physical quantity corresponding to the identified resonance frequency. The image analysis device according to claim 1 .
3. The multi-viewpoint image of the sample is obtained by continuously imaging the sample while turning on and off the microwave at a constant cycle, The fluorescence intensity determination unit determines, as the fluorescence intensity of the quantum sensor, a relative value of the fluorescence intensity of the quantum sensor captured in the multi-viewpoint image when the microwave is on and when the microwave is off, which are successive in time series. The image analysis device according to claim 2.
4. The multi-viewpoint image of the sample is obtained by continuously imaging the sample while turning on and off the microwave at a constant cycle, The image analysis device further includes a fluorescent light beam identification unit that identifies a light beam that becomes bright and dark in the multi-viewpoint image in accordance with an on / off cycle of the microwave as a fluorescent light beam of the quantum sensor, The fluorescence intensity identifying unit determines the intensity of the identified fluorescent light beam. The image analysis device according to claim 2.
5. The fluorescence intensity specifying unit selects, from among a plurality of element images capturing the fluorescent light beam of the quantum sensor, an element image in which the fluorescent light beam is stronger than a certain level, and averages pixel values of the fluorescent light beam of the quantum sensor captured in the selected element image to obtain the fluorescent intensity of the quantum sensor. The image analysis device according to claim 1 .
6. The quantum sensor is a fluorescent nanodiamond.
6. The image analysis device according to claim 1.
7. The physical quantity is temperature.
6. The image analysis device according to claim 1.
8. the quantum sensor is a fluorescent nanodiamond; The physical quantity is temperature.
6. The image analysis device according to claim 1.
9. An image analysis method for analyzing a multi-viewpoint image of a sample including a quantum sensor, the multi-viewpoint image being captured by a compound eye optical system in a state where the sample is irradiated with excitation light having a wavelength that excites the quantum sensor with fluorescence and a microwave having a frequency that causes the quantum sensor to optically detect magnetic resonance, the image analysis method comprising: A step in which a fluorescence intensity determination unit determines the fluorescence intensity of the quantum sensor from the fluorescence light beam of the quantum sensor captured in a plurality of element images in the multi-viewpoint image of the sample; A physical quantity calculation unit calculates a physical quantity indicated by each of the quantum sensors from the fluorescence intensity of the quantum sensor; The image analysis method includes:
10. A program for causing a computer to analyze a multi-viewpoint image of a sample including a quantum sensor, the sample being irradiated with excitation light having a wavelength that excites the quantum sensor with fluorescence and microwaves having a frequency that causes the quantum sensor to optically detect magnetic resonance, the program comprising: a fluorescence intensity determination means for determining the fluorescence intensity of the quantum sensor from the fluorescence light of the quantum sensor captured in a plurality of element images in the multi-viewpoint image of the sample; and a physical quantity calculation means for calculating a physical quantity indicated by the quantum sensor from the fluorescence intensity of the quantum sensor; A program that makes a computer function as a
11. A system for measuring a physical quantity using a quantum sensor, an excitation light generator that generates excitation light of a wavelength that excites the quantum sensor with fluorescence; a microwave generator that generates microwaves at a frequency at which the quantum sensor optically detects magnetic resonance; A compound eye optical system that captures a multi-viewpoint image of a subject; The image analyzing device according to any one of claims 1 to 5, which analyzes a multi-viewpoint image of the sample, the multi-viewpoint image being captured by the compound eye optical system in a state where the sample including the quantum sensor is irradiated with the excitation light and the microwave; A compound-eye optical system quantum metrology system equipped with
12. A method for measuring a physical quantity using a quantum sensor, comprising: A compound eye optical system captures a multi-view image of the sample in a state where a sample including the quantum sensor is irradiated with excitation light having a wavelength that excites the quantum sensor with fluorescence and microwaves having a frequency that causes the quantum sensor to optically detect magnetic resonance; 6. The image analyzing apparatus according to claim 1, further comprising: A compound eye optical system quantum measurement method comprising: