Image processing method and digital assay system

Abstract

To enable high-sensitivity detection of particles in a chamber using a simple imaging device.SOLUTION: A digital assay system 1 comprises a microchamber chip 2, an imaging device 3 for imaging the microchamber chip 2, and an image processing device 4 for processing an image obtained by the imaging device 3. The image processing device 4 comprises an image acquisition unit 41 configured to acquire a plurality of images of the microchamber chip 2 from the imaging device 3, and an averaging unit 42 configured to perform averaging processing on the plurality of images.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for processing images of a microchamber chip. [Background technology]

[0002] Digital assay is a technology that measures the absolute concentration of a substance in a sample by measuring the absolute number of target substances in a solution. Measurements are made using a chip (microchamber chip) equipped with numerous tiny spaces (microchambers) that confine the target substance. When the concentration of the target substance in the sample is sufficiently low, a maximum of one target molecule can be confined in each microchamber. Reactions proceed only in these microchambers, and signals derived from fluorescent substances, etc. are obtained. By counting the number of microchambers in which a signal is generated, it is possible to quantify the target substance at the single-molecule level.

[0003] Most conventional microchamber-based devices are designed to be observed using sophisticated fluorescence microscopes (or devices with equivalent functions) installed in laboratories or specialized facilities. This has led to problems such as the high cost of the microchamber imaging devices, their low portability, and the need for familiarity with their operation. Furthermore, there have been few methods available for ordinary people without specialized skills to determine the presence or absence or concentration of important molecules related to health, such as pathogenic viruses, without relying on specialized facilities such as hospitals.

[0004] In response to this, Non-Patent Document 1 discloses a technique for photographing a microchamber through an aspherical lens using a simple imaging device such as a smartphone. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Y. Minagawa and 3 others, "Mobile imaging platform for digital influenza virus counting", ROYAL SOCIETY OF CHEMISTRY, Lab Chip, 2019, 19, 2678-2687 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technology in Non-Patent Document 1 only achieves 60% of the detection sensitivity of influenza virus particles compared to when a fluorescence microscope is used. This is because the signal-to-noise ratio of images captured by a smartphone is lower than that of a fluorescence microscope.

[0007] The present invention has been made in view of the above problems, and an object of the present invention is to detect particles in a chamber with high sensitivity using a simple imaging device. [Means for solving the problem]

[0008] In order to solve the above problems, the present invention includes the following aspects. Section 1. an imaging step of imaging the microchamber chip by an imaging device; an image acquisition step of acquiring a plurality of images of the microchamber chip from the imaging device; an averaging step of performing averaging processing on the plurality of images; An image processing method comprising: Section 2. Item 2. The image processing method according to item 1, wherein in the averaging step, averaging is performed on 5 to 10 images. Section 3. Item 10. The image processing method according to item 1, wherein the imaging device is installed in a smartphone. Section 4. Item 2. The image processing method according to item 1, wherein the image is an image in RAW data format. Section 5. 5. The image processing method according to any one of items 1 to 4, wherein in the imaging step, the microchamber chip is irradiated with excitation light. Section 6. Item 6. The image processing method according to item 5, further comprising a detection step of detecting, based on the image subjected to the averaging process, a chamber that emits fluorescence in response to the excitation light among a plurality of chambers provided in the microchamber chip. Section 7. A digital assay system comprising: a microchamber chip; an imaging device for imaging the microchamber chip; and an image processing device for processing an image obtained by the imaging device, The image processing device includes: an image acquisition unit that acquires a plurality of images of the microchamber chip from the imaging device; an averaging processing unit that performs averaging processing on the plurality of images; A digital assay system comprising: [Effects of the Invention]

[0009] According to the present invention, particles in a chamber can be detected with high sensitivity using a simple imaging device. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing the configuration of a digital assay system according to one embodiment of the present invention. [Figure 2] 1 is a schematic cross-sectional view of a microchamber chip, an imaging device, and peripheral elements of the microchamber chip. [Figure 3] FIG. 1 is a schematic cross-sectional view of a microchamber chip. [Figure 4] 1 is a flowchart showing a processing procedure of an image processing method according to an embodiment of the present invention. [Figure 5] This is an example of a RAW image. [Figure 6] FIG. 6 is a partially enlarged image of FIG. 5. [Figure 7]This is an image of the same area as in Figure 6 taken using a fluorescence microscope. [Figure 8] 10 is an example of an image obtained by performing averaging processing on a RAW image. [Figure 9] This is an image obtained by subjecting the RAW image shown in FIG. 6 to development processing. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. Note that the present invention is not limited to the following embodiment, and various modifications are possible without departing from the spirit of the present invention.

[0012] (Overall composition) 1 is a block diagram showing the configuration of a digital assay system 1 according to one embodiment of the present invention. The digital assay system 1 includes a microchamber chip 2, an imaging device 3 that images the microchamber chip 2, and an image processing device 4 that processes the image obtained by the imaging device 3.

[0013] 2 is a schematic cross-sectional view of the microchamber chip 2, the imaging device 3, and peripheral elements of the microchamber chip 2. The microchamber chip 2 is provided in a dark box 20 with a light-shielding function. In addition to the microchamber chip 2, the dark box 20 accommodates a detection device including an excitation light source 21, an excitation light filter 22, a condenser lens 23, and a fluorescence filter 24.

[0014] An imaging device 3 is placed on the dark box 20. In this embodiment, the imaging device 3 is an image sensor mounted on a smartphone, but is not limited to this and may be a digital camera. An imaging lens 25 is attached to the top surface of the dark box 20, and the imaging lens 25 faces a lens 31 of the imaging device 3. The structure of the imaging lens 25 is not particularly limited as long as it can magnify the microchamber chip 2 at a sufficient magnification.

[0015] As shown in FIG. 3(a), the microchamber chip 2 is provided with a plurality of microchambers 2a (microspaces) for accommodating particles P1 to be detected. In this embodiment, the particles P1 are influenza viruses. When a hydrophilic solution L1 containing particles P1 and a chromogenic substrate P2 is injected into the microchamber chip 2, the particles P1 are introduced into the microchambers 2a one by one together with the chromogenic substrate P2. Thereafter, as shown in FIG. 3(b), when a hydrophobic solution L2 such as oil is injected into the microchamber chip 2, the particles P1 are confined in the microchambers 2a. A portion of the chromogenic substrate P2 confined in the microchambers 2a together with the particles P1 reacts with the particles P1 to become a fluorescent substance P3.

[0016] The excitation light emitted from the excitation light source 21 shown in FIG. 2 passes through the excitation light filter 22 and the condenser lens 23 and enters the microchamber chip 2. This excites the fluorescent substance P3 in the microchamber 2a shown in FIG. 3(b) and causes it to emit fluorescence R. That is, the fluorescence R is emitted only from the microchamber 2a in which the particle P1 is confined. The fluorescence R emitted from the microchamber 2a passes through the fluorescence filter 24 and the imaging lens 25 shown in FIG. 2 and enters the lens 31 of the imaging device 3.

[0017] The imaging device 3 continuously captures images of the microchamber chip 2 multiple times while the excitation light source 21 is emitting excitation light. In typical smartphones, the raw data acquired by the imaging device is subjected to color adjustment processing as needed, followed by lossy compression processing to significantly reduce the data size and save it in a general-purpose, small-capacity format, such as an 8-bit RGB mode JPG image. Depending on the color correction and compression processing method used, information about the faint fluorescence from the microchambers may be lost. Therefore, when analyzing microchamber chips using reduced-size images, it is essential to use appropriate color correction parameters and compression methods that do not substantially affect the analysis results.

[0018] On the other hand, in this embodiment, raw data acquired by the imaging device 3 may be stored in a RAW data format that retains the raw data intact. RAW data is data acquired by an imaging element that has been compressed to a degree that maintains complete losslessness. In many smartphones and digital cameras, the bit depth of the image information in RAW data is 12-, 14-, or 16-bit. Furthermore, RAW data may also contain supplemental information, such as parameter settings of the imaging device, in addition to image information. While the raw data may be transmitted directly from the imaging device 3 to the image processing device 4 shown in FIG. 1, it is preferable to transmit either appropriately compressed image data or RAW data that has only been losslessly compressed from the imaging device 3 to the image processing device 4. Hereinafter, "image" refers to either image data such as JPG that has been appropriately lossy compressed, or RAW data that maintains losslessness.

[0019] The image processing device 4 is communicably connected to the imaging device 3 and is provided on the cloud in this embodiment. The image processing device 4 can be configured as a general-purpose computer, and includes, as its hardware configuration, an auxiliary storage device 40 such as an HDD or SSD, a main storage device such as a DRAM or SRAM (not shown), and a processor such as a CPU or GPU (not shown). The auxiliary storage device 40 stores a program P and the like.

[0020] The image processing device 4 includes, as functional blocks, an image acquisition unit 41, an averaging processing unit 42, a noise removal unit 43, and a detection unit 44. These functional blocks are realized by the processor of the image processing device 4 reading a program P into a main storage device and executing it. The program P may be downloaded to the image processing device 4 via a communication network such as the Internet, or may be recorded in advance on a computer-readable non-transitory recording medium such as a CD-ROM or an SD card and installed in the image processing device 4 via the recording medium.

[0021] (Image processing procedure) 4 is a flowchart showing the processing steps of the image processing method according to this embodiment, which includes steps S1 to S8.

[0022] In step S1, the digital assay system 1 is set up. Specifically, the microchamber chip 2 is set in a dark box 20 that houses a detection device (excitation light source 21 and optical system), and a smartphone equipped with an imaging device 3 is placed on the dark box 20.

[0023] In step S2, if necessary, the focus of the imaging device 3 is adjusted. In this embodiment, a marker such as a fluorescent bead is placed inside a microchamber 2a of the microchamber chip 2, and the lens 31 of the imaging device 3 is adjusted so that the marker is clearly visible.

[0024] In step S3 (imaging step), the microchamber chip 2 is imaged by the imaging device 3. In this embodiment, while the excitation light source 21 is emitting excitation light, the imaging device 3 continuously images the microchamber chip 2 in a time lapse manner to generate a plurality of images. The number of images is not particularly limited, but is preferably 5 to 50, for example, and more preferably 5 to 10.

[0025] In step S4 (image acquisition step), the image acquisition unit 41 of the image processing device 4 acquires a plurality of images of the microchamber chip 2 from the imaging device 3. That is, the imaging device 3 transmits the RAW images to the image processing device 4 without performing irreversible compression processing on them.

[0026] In step S5 (averaging step), the averaging processor 42 of the image processing device 4 performs averaging on the multiple images. That is, the averaging processor 42 generates an image by averaging the pixel values ​​(brightness values) of each pixel of the multiple images. This makes it possible to visualize the fluorescence emitted from the microchamber 2a, as will be described later.

[0027] In step S6, the noise removal unit 43 of the image processing device 4 performs noise removal processing on the image that has been subjected to the averaging processing as necessary. Specifically, impurities and marks used for focusing are removed based on the color and shape of bright spots in the image.

[0028] In step S7 (detection step), the detection unit 44 of the image processing device 4 detects fluorescence from fluorescent substances generated by the presence of a detection target substance among the multiple microchambers 2a provided in the microchamber chip 2, based on the image that has been subjected to averaging processing. In this embodiment, the detection unit 44 detects bright spots corresponding to the fluorescence and identifies the number of microchambers 2a in which influenza virus particles P1 (FIG. 3) are confined. This makes it possible to predict whether the patient is positive or negative for influenza.

[0029] Note that step S8 may be performed by a human being. As described above, the fluorescence emitted from the microchamber 2a can be visualized by the averaging process, so that bright spots corresponding to the fluorescence can be detected with high accuracy even by human visual inspection.

[0030] (Summary) In this embodiment, the fluorescent chambers can be visualized by averaging multiple images of the microchamber chip 2 captured using the imaging device 3 mounted on a smartphone. Therefore, there is no need to use a costly imaging device such as a fluorescence microscope, and particles in the microchamber 2a can be detected with high sensitivity using the simple imaging device 3.

[0031] (Additional notes) The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope obvious to those skilled in the art. Forms obtained by appropriately combining the technical means disclosed in the embodiments are also included in the technical scope of the present invention.

[0032] In the above embodiment, the particle P1 to be detected is an influenza virus, but it may be another virus. Furthermore, the particle P1 may be a nucleic acid, a protein, a sugar, a lipid, a complex thereof, or the like.

[0033] Furthermore, in the above embodiment, the imaging device 3 continuously captures images of the microchamber chip 2 in a time lapse manner, but a plurality of RAW images may be acquired by capturing a moving image of the microchamber chip 2.

[0034] In addition, in the above embodiment, images captured by the imaging device 3 are processed by the image processing device 4 on the cloud, but a terminal device such as a smartphone equipped with the imaging device 3 may also be given at least some of the functions of the image processing device 4.

[0035] It is also possible to extend the exposure time of the imaging device instead of performing averaging processing, but in the imaging devices installed in general smartphones, extending the exposure time would cause the signal from the light receiving element to reach saturation, so this cannot be used in the present invention. [Example]

[0036] Examples of the present invention will be described below, but the present invention is not limited to the following examples.

[0037] In this example, a solution L1 containing influenza viruses as particles P1 was injected into the microchamber chip 2, and then the influenza viruses were confined within the microchamber 2a by sealing with tape using an adhesive tape method. A camera mounted on a smartphone (iPhone 8) was used as the imaging device 3. The microchamber chip 2 was set in a dark box 20, and the smartphone was placed on the dark box 20 and focused. Then, while irradiating the microchamber chip 2 with excitation light, the imaging device 3 continuously captured images of the microchamber chip 2, obtaining 40 RAW images. These RAW images were converted to tiff format, and the 10 RAW images were then averaged.

[0038] As a comparative example, the same microchamber chip 2 was imaged using a fluorescence microscope.

[0039] The RAW images captured by the imaging device 3 had almost the same image quality. Fig. 5 is a RAW image of a certain frame, and Fig. 6 is an enlarged image of the portion indicated by the rectangular frame in Fig. 5.

[0040] Figure 7 is an image taken using a fluorescence microscope of the same area as in Figure 6. In Figure 7, the triangle on the left indicates a fluorescent bead (marker), and the triangle on the right indicates a chamber (hereinafter referred to as the "fluorescent chamber") that emits fluorescence due to the influenza virus being trapped therein.

[0041] In Figure 6, the fluorescent beads (triangles on the left) are visible, but the fluorescent chambers (triangles on the right) are not.

[0042] Figure 8 is an enlarged image obtained by averaging the 10 raw images from frames 31 to 40, and corresponds to the same region as in Figures 6 and 7. In Figure 8, it can be seen that the fluorescent chamber (indicated by the triangle on the right) is also visualized.

[0043] Figure 9 is an image obtained by processing the RAW image shown in Figure 6. Because the amount of data is significantly reduced by the processing, the fluorescent chambers (triangles on the right) cannot be visualized, and the fluorescent beads (triangles on the left) are also less clear than in the RAW image.

[0044] From the above, it was found that the chamber in which the influenza virus is trapped can be visualized by averaging multiple RAW images of the microchamber chip 2 taken using the imaging device 3 mounted on a smartphone. [Explanation of symbols]

[0045] 1. Digital Assay System 2. Microchamber chip 2a Microchamber 3. Imaging device 4. Image processing device 20 dark box 21 Excitation light source 22 Excitation light filter 23 Condenser lens 24 Fluorescence Filter 25 Imaging lens 31 Lens 40 Auxiliary storage 41 Image acquisition unit 42 Averaging processing section 43 Noise removal section 44 Detector L1 solution L2 Hydrophobic solution P Program P1 particle P2 chromogenic substrate P3 fluorescent material R fluorescence

Claims

1. an imaging step of imaging the microchamber chip by an imaging device; an image acquisition step of acquiring a plurality of images of the microchamber chip from the imaging device; an averaging step of performing averaging processing on the plurality of images; An image processing method comprising:

2. 2. The image processing method according to claim 1, wherein the averaging step performs averaging on 5 to 10 images.

3. The image processing method according to claim 1 , wherein the imaging device is installed in a smartphone.

4. 2. The image processing method according to claim 1, wherein the image is an image in a RAW data format.

5. 5. The image processing method according to claim 1, wherein the microchamber chip is irradiated with excitation light in the imaging step.

6. 6. The image processing method according to claim 5, further comprising a detection step of detecting, based on the image subjected to the averaging process, a chamber that emits fluorescence in response to the excitation light, among a plurality of chambers provided in the microchamber chip.

7. A digital assay system comprising: a microchamber chip; an imaging device for imaging the microchamber chip; and an image processing device for processing an image obtained by the imaging device, The image processing device includes: an image acquisition unit that acquires a plurality of images of the microchamber chip from the imaging device; an averaging processing unit that performs averaging processing on the plurality of images; A digital assay system comprising:

Images