camera device

Abstract

The camera device includes a photocathode having a photoelectric surface, a solid-state imaging element that detects photoelectrons emitted from the photoelectric surface and outputs a detection signal, a vacuum housing that holds the photocathode and the solid-state imaging element in a vacuum state, and a signal processing section that processes the detection signal output from the solid-state imaging element. The solid-state imaging element detects photoelectrons multiplied by electron avalanche multiplication by accelerated photoelectron incidence and outputs a detection signal, and the signal processing section binarizes the detection signal of each frame based on a first threshold value related to a signal intensity set in advance in a manner capable of recognizing the presence or absence of a photon, and performs photon counting by accumulating the binarized signals of the frames after the binarization.

Description

Technical Field

[0001] This disclosure relates to a camera device. Background Technology

[0002] Patent Document 1 describes a method for using a filter in fluorescence detection to prevent the excitation and reflection light from being affected. Patent Document 2 describes a method for synchronizing the timing of excitation / reflection light generation and readout in fluorescence detection to prevent the excitation and reflection light from being affected.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 9-140714

[0006] Patent Document 2: Japanese Patent Application Publication No. 2001-190489 Summary of the Invention

[0007] The technical problem that the invention aims to solve

[0008] In the aforementioned fluorescence detection and bioluminescence observation with even lower signal levels, a further increase in the S / N ratio is required.

[0009] This disclosure was made in view of the above-mentioned circumstances, and its purpose is to provide a camera device that can further improve the S / N ratio.

[0010] Technical means to solve the problem

[0011] One aspect of the camera device disclosed herein is, [1] "a camera device comprising: a photoelectric conversion unit having a photoelectric surface; a solid-state camera element that detects photoelectrons emitted from the photoelectric surface and outputs a detection signal; a vacuum housing that holds the photoelectric conversion unit and the solid-state camera element in a vacuum state; and a signal processing unit that processes the detection signal output from the solid-state camera element, wherein the solid-state camera element detects photoelectrons that are multiplied by electron bombardment by incident accelerated photoelectrons and outputs a detection signal, and the signal processing unit performs binarization processing on the detection signal of each frame based on a first threshold relating to the signal strength preset in a manner that enables the identification of the presence or absence of photons, and performs photon counting by accumulating the binarized signals of each frame after binarization processing."

[0012] In one aspect of the imaging apparatus of this disclosure, a photoelectric conversion unit and a solid-state imaging element are held in a vacuum state. Photoelectrons multiplied by electron bombardment are detected in the solid-state imaging element, and a detection signal is output. Thus, by detecting photoelectrons multiplied by electron bombardment in a vacuum state, a detection signal with minimal multiplication fluctuation can be output. Therefore, in one aspect of the imaging apparatus of this disclosure, the detection signal is binarized based on a first threshold set in a manner capable of identifying the presence or absence of photons, and the binarized signals of each frame are accumulated to perform photon counting. As described above, since the detection signal has minimal multiplication fluctuation and the signal component without noise is appropriately multiplied, the presence or absence of photons can be appropriately identified through binarization processing. Therefore, by accumulating the binarized signals with appropriately identified photon presence or absence to perform photon counting, an image with an improved signal-to-noise ratio (S / N ratio) can be acquired. As described above, according to one aspect of the imaging apparatus of this disclosure, a further improvement in the S / N ratio can be achieved.

[0013] One aspect of the imaging apparatus disclosed herein may also be, [2] "the imaging apparatus as described in [1], wherein, when the number of frames accumulated for an object reaches a preset upper limit frame number, the signal processing unit deletes the binarized signal of the oldest frame when accumulating the binarized signal of a new frame." Based on this structure, the signal of the latest frame (the frame with the upper limit frame number) can always be acquired. Thus, for example, in bioluminescence observation, information about the motion of an object (movement information) can be acquired.

[0014] One aspect of the imaging apparatus disclosed herein may also be, [3] "the imaging apparatus as described in [2], wherein the signal processing unit performs moving average processing or accumulation processing on the detection signal of a predetermined exclusion frame without performing the binarization processing." According to this structure, signals involving light of different properties can be acquired, for example, through moving average processing or accumulation processing, separately from the aforementioned binarization signal accumulation processing. Thus, it is possible to acquire both signals simultaneously by switching between signals involving light of different properties.

[0015] One aspect of the imaging apparatus disclosed herein may also be, [4] "the imaging apparatus as described in any one of [1] to [3], wherein the signal processing unit multiplies the signal obtained by accumulating the binarized signals of each frame by a predetermined constant." For example, in bioluminescence observation, the signal quantity is extremely small, so when accumulating the binarized signals of each frame to generate one image, the signal quantity may sometimes be insufficient. In this regard, by multiplying the signal obtained by accumulating the binarized signals by a constant, the signal quantity increases, thereby improving the S / N ratio and generating image contrast. Furthermore, this processing is possible precisely because the present method can appropriately separate noise components and signal components through binarization processing. That is, when the signal contains a large amount of noise components, even multiplying by a constant cannot improve the S / N ratio, but in the present method, since the signal contains almost no noise components through binarization processing, multiplying by a constant can increase only the signal components, thereby improving the S / N ratio.

[0016] One aspect of the imaging apparatus disclosed herein may also be, [5] "the imaging apparatus as described in any one of [1] to [4], wherein the signal processing unit determines the number of photons based on a second threshold relating to the signal strength, which is preset in a manner that enables the identification of the number of photons." In this way, in addition to the first threshold used to identify the presence or absence of photons, a second threshold is set in a manner that enables the identification of the number of photons, and by determining the number of photons based on the second threshold, a signal containing two or more photons can be identified. Furthermore, this processing is possible precisely because the signal multiplication fluctuation is small in this method.

[0017] One aspect of the camera apparatus disclosed herein may also be, [6] "the camera apparatus as described in any one of [1] to [5], wherein the signal processing unit excludes frames with abnormal photon counts from the accumulation object based on a third threshold related to the signal strength preset in a way that can identify abnormal photon counts." For example, in bioluminescence, the amount of light emitted is extremely small, so there is almost no signal with a large number of photons (e.g., more than 3 photons) entering in one frame. In the case of detecting such a large number of photons, there is a high probability that it is an anomaly such as cosmic ray noise. By setting a third threshold to regard a signal strength above a certain point as an anomaly and not counting it, the influence of abnormal noise can be eliminated. Furthermore, this processing is possible precisely because the signal multiplication fluctuation is small in this method.

[0018] The effects of the invention

[0019] According to the camera device disclosed herein, it is possible to further improve the S / N ratio. Attached Figure Description

[0020] Figure 1 This is a diagram schematically showing the structure of the camera device according to the first embodiment.

[0021] Figure 2 This is a cross-sectional view of an electron bombardment imaging sensor.

[0022] Figure 3 (a) is a graph representing the signal detected in sCMOS. Figure 3 (b) is a graph representing the signal detected in EB-CMOS.

[0023] Figure 4 This is a diagram illustrating the characteristics of an electron bombardment camera compared to sCMOS and cooled CCD cameras.

[0024] Figure 5 This is a diagram illustrating the features of an electronic bombardment camera compared to a camera device that combines an image intensifier and a camera element.

[0025] Figure 6 (a) is a graph representing a live image. Figure 6 (b) is a graph representing the moving average image. Figure 6 (c) is a diagram representing the photon counting image.

[0026] Figure 7 This is a diagram illustrating an example of setting a second threshold.

[0027] Figure 8 This is a diagram illustrating an example of setting a third threshold.

[0028] Figure 9 This is a diagram schematically illustrating the structure of a modified camera device.

[0029] Figure 10 This is a diagram schematically illustrating the structure of a modified camera device.

[0030] Figure 11 This is a diagram schematically illustrating the structure of the camera device according to the second embodiment.

[0031] Figure 12 This is a diagram illustrating the rolling photon counting method.

[0032] Figure 13 (a) is a schematic diagram of acquiring motion information using a camera device. Figure 13 (b) is a schematic diagram of acquiring motion information using a cooled CCD.

[0033] Figure 14 This is a diagram illustrating the rolling photon counting of a modified example.

[0034] Figure 15 This is a diagram schematically illustrating the structure of a modified camera device.

[0035] Figure 16This is a diagram schematically illustrating the structure of a modified camera device.

[0036] Figure 17 This is a diagram schematically illustrating the structure of a modified camera device.

[0037] Figure 18 This is a schematic diagram illustrating the structure for fluorescence observation in the imaging device of the third embodiment.

[0038] Figure 19 This diagram illustrates the switching between rolling photon counting, rolling averaging, and cumulative processing methods.

[0039] Figure 20 (a) is a graph showing the observation results of bioluminescence. Figure 20 (b) is a graph showing the observation results of fluorescence. Detailed Implementation

[0040] [First Implementation Method]

[0041] Hereinafter, the first embodiment of the present invention will be described in detail with reference to the accompanying drawings. Furthermore, in the various drawings, the same or equivalent parts are labeled with the same reference numerals, and repeated descriptions are omitted.

[0042] Figure 1 This diagram schematically illustrates the structure of the imaging device 1 according to the first embodiment. The imaging device 1 is, for example, a device for observing bioluminescence with extremely low signal intensity. Besides bioluminescence, other luminescent phenomena include fluorescence and chemiluminescence. Bioluminescence refers to the phenomenon where organisms produce and emit light. Fluorescence refers to the phenomenon where organisms, upon irradiation by excitation light, absorb energy and become excited, emitting light. Chemiluminescence refers to the phenomenon where organisms, upon becoming excited by a chemical reaction, emit light. For example, in fluorescence observation, increasing the excitation intensity strengthens the fluorescence intensity; however, this can lead to damage to the organism (cells, etc.), increased autofluorescence, and a potential decrease in quantification during observation. In this respect, bioluminescence observation does not require excitation light, avoids photodamage to the organism (cells), and allows for long-term imaging. Furthermore, in bioluminescence observation, since background light from autofluorescence is suppressed, only signal values ​​can be stably acquired, ensuring quantification during observation. On the other hand, in bioluminescence observation, the luminescence (signal intensity) is extremely weak, sometimes making it difficult to improve the signal-to-noise ratio (S / N ratio). The imaging device 1 of this embodiment employs a method that can improve the S / N ratio even in such weak bioluminescence observation (details will be described later).

[0043] like Figure 1As shown, the imaging device 1 includes an objective lens 2, a filter 3, an electron bombardment camera 10, a high-voltage controller 20, a PC (personal computer) 30, and a monitor 40. The following description will follow the observation of bioluminescence from cell S using the imaging device 1.

[0044] Objective lens 2 is a lens that focuses the bioluminescence from cell S onto the electron bombardment imaging sensor 11 of electron bombardment camera 10. The bioluminescence focused by objective lens 2 is incident on electron bombardment imaging sensor 11 of electron bombardment camera 10 through filter 3.

[0045] The electron bombardment camera 10 includes an electron bombardment imaging sensor 11, a drive unit 12, a control unit 13, a high-voltage power supply 14, a signal processing unit 15, and an interface 16.

[0046] like Figure 2 As shown, the electron bombardment imaging sensor 11 includes a photocathode 111 (photoelectric conversion unit), a solid-state imaging element 112 with electron bombardment multiplication function, and a vacuum housing 113. In this embodiment, the solid-state imaging element 112 is an EB-CMOS (Electron Bombarded-Complementary Metal-Oxide-Semiconductor). The vacuum housing 113 is a housing that holds the photocathode 111 and the solid-state imaging element 112 in a vacuum state. The vacuum housing 113 has an entrance window 113a for incident light focused by the objective lens 2. Light incident from the entrance window 113a reaches the photoelectric surface 111a of the photocathode 111.

[0047] The photocathode 111 is a photoelectric conversion unit having a photoelectric surface 111a. The photoelectric surface 111a converts incident light into photoelectrons. The photocathode 111 is fabricated, for example, in the form of a semiconductor layer, and can be a crystalline photoelectric surface with a single crystal structure (e.g., a III-V semiconductor such as GaAs, or a II-VI semiconductor such as CdTe). Alternatively, the photocathode 111 can also be an alkaline photoelectric surface with a polycrystalline structure such as SbK₂Cs, SbRb₂Cs, SbCs₃, SbNa₃, SbNaKRbCs, SbNaKCs, or SbNa₂KCs. The photocathode 111 is electrically connected to an electrode 118, and a predetermined potential is applied to the photocathode 111 (photoelectric surface 111a) via the electrode 118. Photoelectrons released from the photoelectric surface 111a of the photocathode 111 are emitted into the vacuum and detected by a solid-state imaging element 112.

[0048] The solid-state imaging element 112 is an imaging element that detects photoelectrons emitted from the photoelectric surface 111a and outputs a detection signal. The solid-state imaging element 112 detects photoelectrons accelerated by incident light through electron bombardment and multiplication, generated by energy corresponding to the potential difference between the photocathode 111 and the solid-state imaging element, and outputs a detection signal. The solid-state imaging element 112 is electrically connected to pin 120 via bonding wire 119. The solid-state imaging element 112 is positioned opposite the photocathode 111 and is located within the vacuum housing 113 on the opposite side of the end where the photocathode 111 is located. The solid-state imaging element 112 may be, for example, a back-side illumination (BSI) CMOS sensor. The detection signal output from the solid-state imaging element 112 is input to a signal processing unit 15 located outside the vacuum housing 113. The voltage applied to the solid-state imaging element 112 is approximately proportional to the gain obtained as the detection signal. The desired gain can be obtained by adjusting the potential difference between the photoelectric surface 111a of the photocathode 111 and the solid-state imaging element 112. In addition, the EB-CMOS (solid-state imaging element 112) can also be replaced by the EB-CCD (Electron-Bombarded Charge-Coupled Device).

[0049] Figure 3 (a) is a graph representing the signal detected in sCMOS (scientific CMOS). Figure 3 (b) is a graph representing the signal detected in EB-CMOS. Figure 3 The sCMOS in (a) is illustrated as a structure for comparison with the electron bombardment camera 10. Figure 3 In (a) and (b), the horizontal axis represents time, and the vertical axis represents the semaphore. Figure 3 In (a) and (b), the smaller signal amounts represent readout noise and noise such as dark current generated within the solid-state imaging element 112. Figure 3 In the sCMOS shown in (a), dark current is suppressed and readout noise is reduced through cooling, thereby reducing noise and improving the signal-to-noise ratio. On the other hand, sCMOS does not have a multiplication function and cannot increase the signal components. In this respect, as Figure 3 As shown in (b), in EB-CMOS, the signal components that do not contain noise (readout noise, dark current noise) originating from the driving of the solid-state imaging element 112 are multiplied by the above-described gain, thereby improving the S / N ratio. This EB-CMOS can achieve a higher S / N ratio than sCMOS in measurements under ultra-low illumination conditions (e.g., when the average number of photons detected per pixel per frame is lower than the photon counting region of a single photon).

[0050] Back Figure 1 The drive unit 12 is a circuit for driving the electron bombardment imaging sensor 11. The control unit 13 is a structure for changing various settings in the electron bombardment imaging sensor 11, such as changing the number of pixels and the field of view through binning, subarray, and other processes. The high-voltage power supply 14 is a power supply applied to the photoelectric surface 111a and the solid-state imaging element 112 used to generate electron bombardment multiplication gain. The voltage of the high-voltage power supply 14 is regulated by the high-voltage controller 20.

[0051] The signal processing unit 15 processes the detection signal output from the solid-state imaging element 112. Based on a first threshold related to the signal strength preset in a way that can identify the presence or absence of photons, the signal processing unit 15 performs binarization processing on the detection signal of each frame, and performs photon counting by accumulating the binarized signals of each frame.

[0052] like Figure 1 As shown, the signal processing unit 15 outputs a real-time signal (Live signal), a binarized signal after binarization processing, and a photon counting signal obtained by accumulating the binarized signals. The real-time signal refers to the raw signal for which no special processing has been performed. Such a real-time signal is used, for example, when acquiring a moving average image (rolling average image).

[0053] Reference Figure 4 and Figure 5 The binarization process will be explained. The signal processing unit 15 considers the brightness value (signal intensity) when a photon is incident and pre-sets a brightness value capable of identifying the presence or absence of a photon as a first threshold. This first threshold is set to a brightness value that is conceived to be impossible to reach in the case of "no photon" (no noise only) and is guaranteed to be reached in the case of "photon presence." By performing binarization based on whether such a first threshold is exceeded, the presence or absence of photons in each frame can be appropriately identified. Figure 4 As shown in the bottom paragraph, the signal processing unit 15 determines that frames with signal strength greater than the first threshold are "photon present" and sets them to "1", while frames with signal strength less than the first threshold are determined to be "no photon present" and set to "0". Figure 4 As shown, the signal contains dark current and readout noise, but as mentioned above, the first threshold is set to a brightness value that is impossible to reach in the case of noise only with "no photons", so that the presence or absence of photons can be properly identified even when noise is taken into account.

[0054] Figure 4This diagram illustrates the features of an electron bombardment camera 10 compared to an sCMOS camera and a cooled CCD camera. Because an sCMOS camera lacks a multiplication function, it cannot multiply signal components alone, and therefore cannot identify the presence or absence of photons based on the aforementioned first threshold. Similarly, a cooled CCD camera cannot increase signal components alone compared to noise such as readout noise, and therefore also cannot identify the presence or absence of photons based on the aforementioned first threshold. In contrast, in the electron bombardment camera 10 of this embodiment, only signal components are multiplied, thus enabling appropriate identification of the presence or absence of photons based on the first threshold.

[0055] Figure 5 This diagram illustrates the features of an electron bombardment camera 10, compared to an imaging device that combines an image intensifier and an imaging element (CMOS camera). Since an image intensifier exists as a sensor with signal multiplication capabilities, a structure for an imaging device that uses a CMOS camera to capture the light signal multiplied and output by the image intensifier can be considered. However, as... Figure 5 As shown in the image above, the light signal amplified by the image intensifier fluctuates greatly. When using a CMOS camera for imaging, it may sometimes be impossible to properly identify the presence or absence of photons based on the first threshold (inability to properly binarize). Without proper binarization, there is a risk of acquiring a coarse image with a low signal-to-noise ratio (S / N). In this regard, as... Figure 5 As shown in the figure below, in the electron bombardment camera 10, because it detects photoelectrons that have multiplied through electron bombardment in a vacuum state, the multiplication fluctuation of the detection signal is small, and the presence or absence of photons can be appropriately identified based on the first threshold. Furthermore, in Figure 5 The following attached diagrams ( Figure 5 , Figure 7 , Figure 8 In the original text, dark current and readout noise were not distinguished and were represented together.

[0056] Figure 6 (a) is a graph representing an example of a real-time image. Figure 6 (b) is a graph representing an example of a moving average image. Figure 6 (c) is a diagram representing an example of a photon counting image. Figure 6 The images shown in (a) to (c) are examples of images acquired by the electron bombardment camera 10, representing an exposure time of 20 msec and an exposure value of 500 V. Figure 6 The real-time image shown in (a) is an image acquired from a 1-frame real-time signal (the raw signal). Due to the low illumination, almost nothing equivalent to the signal is visible. If such images are summed up, they will result in an image with a very poor signal-to-noise ratio.

[0057] Figure 6The moving average image shown in (b) is an image acquired from a real-time signal. As mentioned above, a suitable image cannot be acquired using a real-time signal, therefore a moving average can be used as a method to improve the signal-to-noise ratio. However, under ultra-low illumination conditions such as bioluminescence, even assuming an accumulation of approximately 15,000 frames, noise components will be acquired proportionally, therefore... Figure 6 As shown in (b), the S / N ratio cannot be significantly improved.

[0058] In contrast, Figure 6 The photon counting image shown in (c) is obtained by eliminating noise components (dark current, readout noise) and accumulating the signal components. Thus, for example, by accumulating the signal over 15,000 frames, only the signal component (S) is accumulated, thereby greatly improving the S / N ratio.

[0059] Back Figure 1 The signal processing unit 15 can also multiply the signal obtained by accumulating the binarized signals of each frame by a predetermined constant. As described above, an image is obtained by accumulating binarized signals of any number of frames. At this time, the darker the emission from the cells S, which are bioluminescent samples, the less signal is generated when the final image is generated. Therefore, as a method for generating image contrast, the signal processing unit 15 multiplies the final signal obtained by accumulating the binarized signals by a predetermined constant. As described above, since noise components and signal components are appropriately separated by binarization processing, multiplying the signal obtained by accumulating the binarized signals by a predetermined constant can increase almost only the signal component. As a result, the S / N ratio in the final image can be improved. Furthermore, the signal processing unit 15 can multiply the final signal obtained by accumulating the binarized signals by a predetermined constant, and it can also multiply the signal at any time by a predetermined constant. For example, in bioluminescence, the emission is extremely weak. When binarizing such a weak signal, in order to confirm whether an appropriate signal has been detected at any stage, it is preferable to multiply the signal at any time by a constant for confirmation. Therefore, the signal processing unit 15 can also multiply the signal at any time by a predetermined constant.

[0060] Furthermore, the signal processing unit 15 can also determine the number of photons based on a second threshold related to the signal strength, which is preset in a way that allows for the identification of photon count. In the aforementioned photon counting based on the first threshold, the presence or absence of photons can be identified. This first threshold can be considered a threshold used to eliminate noise components. In addition, by setting a second threshold as a threshold for identifying the number of photons, it is possible to distinguish between one photon and two photons. Furthermore, in the case of bioluminescence, since it is difficult to imagine that more than three photons would enter in one frame, the second threshold is, for example, set to a brightness value that distinguishes between one photon and two photons. Therefore, as... Figure 7As shown, the second threshold is set to a brightness value that is conceived to be impossible to achieve with "1 photon" but is certain to be achieved with "2 photons". Furthermore, the second threshold is set to a brightness value greater than the first threshold. Whether the number of photons is "1" or "2" can be identified based on whether this second threshold is exceeded.

[0061] Furthermore, the signal processing unit 15 can also exclude frames with abnormal photon counts from the accumulation object based on a third threshold related to the signal strength, which is preset in a way that can identify abnormal photon counts. As mentioned above, in the case of bioluminescence, it is difficult to imagine that more than three photons would enter in one frame. If more than three photons are detected, it is highly likely that it is an anomaly such as cosmic ray noise. Therefore, as Figure 8 As shown, by setting a third threshold to a brightness value that is assumed to be impossible to achieve with fewer than "3 photons" but is certain to be achieved with more than "3 photons," frames with abnormal photon counts (noise) can be appropriately identified. Then, frames with such abnormal photon counts can be appropriately excluded from the accumulation object. Here, the frames in the accumulation object refer to the frames used to construct the image.

[0062] The signal processing unit 15 outputs the real-time signal, the binarized signal, and the photon count signal obtained by summing the binarized signals of each frame to the PC30 via the interface 16. The interface 16 is a device for connecting the signal output from the signal processing unit 15 to a downstream device (here, the PC30).

[0063] PC30 includes an image input board 31 and software 32. The function of the image input board 31 is to visualize the signals from the signal processing unit 15. An example of the result of visualizing each signal is as follows: Figure 6 (a) ~ Figure 6 As shown in (c), software 32 performs various processing related to the image. Software 32 outputs the image to monitor 40. Monitor 40 is a display device for displaying the image.

[0064] (Effects)

[0065] Next, the effects of the camera device 1 in this embodiment will be explained.

[0066] The imaging device 1 of this embodiment includes: a photocathode 111 having a photoelectric surface 111a; a solid-state imaging element 112 that detects photoelectrons emitted from the photoelectric surface 111a and outputs a detection signal; a vacuum housing 113 that holds the photocathode 111 and the solid-state imaging element 112 in a vacuum state; and a signal processing unit 15 that processes the detection signal output from the solid-state imaging element 112. The solid-state imaging element 112 detects photoelectrons that are multiplied by electron bombardment due to the incident accelerated photoelectrons and outputs a detection signal. The signal processing unit 15 performs binarization processing on the detection signal of each frame based on a first threshold related to the signal strength preset in a way that can identify the presence or absence of photons, and performs photon counting by accumulating the binarized signals of each frame.

[0067] In the imaging device 1 of this embodiment, the photocathode 111 and the solid-state imaging element 112 are held in a vacuum state. Photoelectrons multiplied by electron bombardment are detected in the solid-state imaging element 112, and a detection signal is output. Thus, by detecting photoelectrons multiplied by electron bombardment in a vacuum state, a detection signal with minimal multiplication fluctuation can be output. Therefore, in the imaging device 1 of this embodiment, the detection signal is binarized based on a first threshold set in a manner capable of identifying the presence or absence of photons, and the binarized signals of each frame are accumulated to perform photon counting. As described above, since the detection signal has minimal multiplication fluctuation and the signal component without noise is appropriately multiplied, the presence or absence of photons can be appropriately identified through binarization processing. Therefore, by accumulating the binarized signals with appropriately identified photon presence or absence to perform photon counting, an image with an improved signal-to-noise ratio (S / N ratio) can be acquired. As described above, the imaging device 1 of this embodiment can achieve a further improvement in the S / N ratio.

[0068] The signal processing unit 15 can also multiply the signal obtained by accumulating the binarized signals of each frame by a predetermined constant. For example, in bioluminescence observation, the signal quantity is extremely small, so sometimes the signal quantity is insufficient when accumulating the binarized signals of each frame to generate a single image. In this case, by multiplying the signal obtained by accumulating the binarized signals by a constant, the signal quantity increases, thereby improving the S / N ratio and increasing the image contrast. Furthermore, this method is possible because it can appropriately separate noise components and signal components through binarization processing. That is, when the signal contains a large amount of noise, multiplying by a constant will not improve the S / N ratio, but in this method, since the signal contains almost no noise components through binarization processing, multiplying by a constant can increase only the signal components, thereby improving the S / N ratio.

[0069] The signal processing unit 15 can also determine the number of photons based on a second threshold related to the signal strength, which is preset in a way that allows for the identification of the number of photons. In this way, in addition to the first threshold used to identify the presence or absence of photons, a second threshold is set in a way that allows for the identification of the number of photons. By determining the number of photons based on the second threshold, signals containing two or more photons can be identified. Furthermore, this processing is possible precisely because the signal multiplication fluctuations are minimal in this method.

[0070] The signal processing unit 15 can also exclude frames with abnormal photon counts from the accumulation object based on a third threshold related to the signal strength, which is preset in a way that can identify abnormal photon counts. For example, in bioluminescence, the amount of light emitted is extremely small, so there is almost no signal with a large number of photons (e.g., more than 3 photons) entering in a single frame. In the case of detecting such a large number of photons, there is a high probability that it is an anomaly such as cosmic ray noise. By setting a third threshold to regard signals with a certain signal strength or higher as an anomaly and not counting them, the influence of abnormal noise can be eliminated. Furthermore, this processing is possible precisely because the signal multiplication fluctuations are small in this method.

[0071] The camera device 1 of the first embodiment has been described above, but the camera device of the first embodiment is not limited to the above. Figure 9 and Figure 10 This is a diagram schematically illustrating the structure of a modified camera device.

[0072] As a structure equivalent to the aforementioned signal processing unit Figure 9 The imaging device 1A shown includes a first signal processing unit 15A and a second signal processing unit 15B. The first signal processing unit 15A is disposed inside the electron bombardment camera 10. The second signal processing unit 15B is disposed outside the electron bombardment camera 10. The first signal processing unit 15A performs binarization processing on the detection signal output from the electron bombardment imaging sensor 11 based on a first threshold, and outputs a binarized signal. The second signal processing unit 15B acquires the binarized signal output from the first signal processing unit 15A via interface 16, performs photon counting by accumulating the binarized signals of each frame, and outputs a photon count signal. Thus, the signal processing unit can be divided into a structure that performs binarization processing (first signal processing unit 15A) and a structure that performs accumulation processing (second signal processing unit 15B).

[0073] Figure 10The imaging device 1B shown is similar to the imaging device 1A, and as a structure equivalent to a signal processing unit, it includes a first signal processing unit 15A inside the electron bombardment camera 10 and a second signal processing unit 15B outside the electron bombardment camera 10. Thus, in the imaging device 1B, the first signal processing unit 15A only outputs the detection signal (real-time signal) to the next stage, while the second signal processing unit 15B performs binarization and accumulation processing. In this way, the functions undertaken by the two signal processing units can be appropriately selected.

[0074] [Second Implementation]

[0075] Next, refer to Figures 11-13 The second embodiment of this disclosure will be described. In the second embodiment, the differences from the first embodiment will be mainly described, and descriptions that are repeated in the first embodiment will be omitted.

[0076] As a signal processing unit, the camera device 101 of the second embodiment includes a signal processing unit 115. The signal processing unit 115 is configured to perform rolling photon counting in addition to the same processing as the signal processing unit 15 of the first embodiment. The rolling photon counting will be described below.

[0077] In rolling photon counting, when the number of frames to be accumulated reaches a preset upper limit, the signal processing unit 115 deletes the oldest frame's binarized signal when accumulating the binarized signal of a new frame. For example, if 3000 frames (upper limit: 3000) are needed to obtain sufficient signal, the information of the first frame is deleted when the information of the 3001st frame is acquired and accumulated, and the information of the second frame is deleted when the information of the 3002nd frame is acquired and accumulated. By acquiring signals in this way, the latest 3000 frames of signal can always be acquired / accumulated.

[0078] exist Figure 12 The example shown illustrates an assumption that the time (i.e., the upper limit of frames) for one image is 4, and the example shows an example where the oldest frame's binarized signal is deleted whenever the latest frame's binarized signal is acquired. In this way, by always acquiring the latest upper limit of frame binarized signals, information about the object's motion (movement information) can be obtained, for example, in bioluminescence observation.

[0079] Figure 13 (a) is a schematic diagram of acquiring motion information using camera device 101. Figure 13 (b) is a schematic diagram illustrating the acquisition of motion information using a cooled CCD camera. For example... Figure 13As shown in (b), in a cooled CCD camera, a long exposure is required to accumulate the signal, and the image is acquired in one frame only at a specific instant (e.g., an instant at a 5-minute interval). Therefore, an image capturing, for example, the motion of cell S becomes a discontinuous, jumping image. In contrast, as... Figure 13 As shown in (a), in the camera device 101, since the latest image can be acquired at any time by the user, the movement of cell S over time can be captured.

[0080] (Effects)

[0081] Next, the effects of the camera device 101 in this embodiment will be explained.

[0082] In the imaging device 101, the signal processing unit 115 can delete the oldest frame's binarized signal when accumulating the binarized signal of a new frame, provided that the number of frames accumulated for the object has reached a preset upper limit. With this structure, the signal of the latest frame (the frame above the upper limit) can always be acquired. Therefore, for example in bioluminescence observation, by saving the image at any time for the user, information about the object's motion (movement information) can be obtained.

[0083] The camera device 101 of the second embodiment has been described above, but the camera device of the second embodiment is not limited to the above. Figures 14-17 This is a diagram illustrating a modified example of a camera device.

[0084] like Figure 14 As shown, in rolling photon counting, instead of saving frame by frame, a predetermined number of frames can be saved. That is, for example, if 3000 frames are required for image acquisition, the binarized and accumulated images can be saved every 1000 frames, collecting the information required for the 3000 frames. Figure 14 In this method, frames labeled Accumulation I through Accumulation IV are grouped into sets of 1000 frames each, and images can be saved in these frame groups. When repeatedly accumulating and deleting in rolling photon counting, for example, 1000 frames of Accumulation I can be deleted, and a new set of 1000 frames of Accumulation IV can be accumulated and saved. According to this method, compared to saving frame by frame, the number of saved images becomes 1 / 1000, thus reducing the data volume to 1 / 1000.

[0085] As a structure equivalent to the aforementioned signal processing unit Figure 15The imaging device 101A shown includes a first signal processing unit 115A and a second signal processing unit 115B. The first signal processing unit 115A is disposed inside the electron bombardment camera 10. The second signal processing unit 115B is disposed outside the electron bombardment camera 10. The first signal processing unit 115A performs binarization processing on the detection signal output from the electron bombardment imaging sensor 11 based on a first threshold, and outputs a binarized signal after binarization processing. In addition, the first signal processing unit 115A performs photon counting by accumulating the binarized signals of each frame, and outputs a photon count signal. The second signal processing unit 115B performs rolling photon counting, which deletes the binarized signal of the oldest frame when accumulating the binarized signal of a new frame when the number of frames to be accumulated reaches a preset upper limit frame number. In this way, the signal processing unit can be divided into a structure that performs binarization processing and accumulation processing (first signal processing unit 115A) and a structure that performs rolling photon counting processing (second signal processing unit 115B).

[0086] As a structure equivalent to the aforementioned signal processing unit Figure 16 The imaging device 101B shown includes a first signal processing unit 115A and a second signal processing unit 115B. The first signal processing unit 115A is located inside the electron bombardment camera 10. The second signal processing unit 115B is located outside the electron bombardment camera 10. The first signal processing unit 115A performs binarization processing on the detection signal output from the electron bombardment imaging sensor 11 based on a first threshold, and outputs a binarized signal. The second signal processing unit 115B acquires the binarized signal output from the first signal processing unit 15A via interface 16, performs photon counting by accumulating the binarized signals of each frame, and outputs a photon count signal. In addition, the second signal processing unit 115B performs rolling photon counting by deleting the binarized signal of the oldest frame when accumulating the binarized signal of a new frame if the number of frames to be accumulated reaches a preset upper limit frame number. In this way, the functions performed by the two signal processing units can be appropriately selected.

[0087] As a structure equivalent to a signal processing unit Figure 17 The imaging device 101C shown includes a first signal processing unit 115A inside the electron bombardment camera 10 and a second signal processing unit 115B outside the electron bombardment camera 10. Thus, in the imaging device 101C, the first signal processing unit 115A only outputs a detection signal (real-time signal) to the next stage, while the second signal processing unit 15B performs binarization processing, accumulation processing, and photon counting processing. In this way, the functions performed by the two signal processing units can be appropriately selected.

[0088] [Third Implementation Method]

[0089] Next, refer to Figures 18-20The third embodiment of this disclosure will be described. In the third embodiment, the differences from the first and second embodiments will be mainly described, and the descriptions that are repeated in the first and second embodiments will be omitted.

[0090] In the imaging device of the third embodiment, imaging is performed while switching between the aforementioned rolling photon counting and moving average processing or accumulation processing at arbitrary intervals. When performing bioluminescence observation, the signal processing unit of the imaging device can perform moving average processing or accumulation processing on the detection signal of a predetermined exclusion frame without binarization processing, thereby performing fluorescence observation. In this case, the signal processing unit's signal regarding the fluorescence observation quantity is not added to the bioluminescence observation signal. While retaining the bioluminescence signal prior to fluorescence acquisition in memory, the signal processing unit accumulates the bioluminescence signal after fluorescence acquisition, thereby saving the bioluminescence image immediately after fluorescence acquisition.

[0091] Figure 18 This diagram schematically illustrates the structure for fluorescence observation in the imaging device of the third embodiment. In addition to the structures possessed by imaging device 1 and imaging device 101 described above, the imaging device of the third embodiment also possesses... Figure 18 The structure shown is for fluorescence observation. Figure 18 In the example shown, the structure used for fluorescence observation includes a dichroic mirror 501 and an excitation filter 502.

[0092] Excitation filter 502 is an optical element used to extract light of the desired wavelength from an excitation source (not shown, such as a mercury lamp) to excite the fluorescent substance. Excitation filter 502 can be a bandpass filter that only transmits light of a specific wavelength and not other light. Dichroic mirror 501 is an optical element used to separate excitation light and fluorescence. Dichroic mirror 501 reflects the light that has passed through excitation filter 502 and directs it towards cell S, while allowing fluorescence emitted from cell S to pass through and be directed towards filter 3.

[0093] Figure 19 This diagram illustrates the switching between rolling photon counting and rolling averaging methods. Figure 19 In the example shown, the signals of the latest four frames are accumulated during rolling photon counting. However, for frames designated for exclusion during fluorescence observation without photon counting, moving average or accumulation processing is performed instead of binarization for fluorescence observation, and these frames are excluded from the accumulation objects of rolling photon counting. Furthermore, during rolling photon counting, while the bioluminescence signal prior to fluorescence acquisition is stored in memory, the bioluminescence signal after fluorescence acquisition is accumulated, thereby acquiring a bioluminescence image immediately after fluorescence acquisition.

[0094] Figure 20 (a) is a graph showing the observation results of bioluminescence. Figure 20 (b) is a graph showing the observation results of fluorescence. For example... Figure 20 As shown in (a) and (b), the brightness values ​​of fluorescence are much higher than those of bioluminescence. Therefore, if simultaneous observation is performed using, for example, a CCD, the image of bioluminescence would be disrupted, making proper detection impossible. In this regard, fluorescence observation is obtained by using a moving average or cumulative processing separate from rolling photon counting, thereby enabling the simultaneous acquisition of images of bioluminescence and fluorescence while switching between them within the same time period.

[0095] This disclosure is not limited to the embodiments described above. For example, the processing involved in image generation can be completed in the signal processing unit (all implementations). In this case, the aforementioned PC 30 and monitor 40 structures may not be required. Furthermore, the generated image may not necessarily be displayed on the monitor in real time, or it may be recorded by outputting to a USB or the like. With such a structure, the recording of the generated image can be appropriately performed even without the aforementioned PC 30 and monitor 40. In addition, various signal processing operations such as binarization processing may not be performed inside the aforementioned electron bombardment camera 10, but may be performed by software built into the PC, for example.

[0096] Explanation of reference numerals in the attached figures

[0097] 1, 1A, 1B, 101, 101A, 101B, 101C… camera device; 15, 15A, 15B, 115, 115A, 115B… signal processing unit; 111… photocathode (photoelectric conversion unit); 111a… photoelectric surface; 112… solid-state imaging element; 113… vacuum housing.

Claims

1. A camera device, wherein, have: A photoelectric conversion unit with a photoelectric surface; A solid-state imaging element that detects photoelectrons emitted from the photoelectric surface and outputs a detection signal; The vacuum housing of the photoelectric conversion unit and the solid-state imaging element is maintained under vacuum conditions; and A signal processing unit that processes the detection signal output from the solid-state imaging element. The solid-state imaging element detects photoelectrons that are accelerated by incident light and multiplied by electron bombardment, and outputs the detection signal. The signal processing unit performs binarization processing on the detection signal of each frame based on a first threshold related to the signal strength that is preset in a way that can identify the presence or absence of photons, and performs photon counting by accumulating the binarized signals of each frame.

2. The camera device as claimed in claim 1, wherein, When the number of frames to be accumulated reaches a preset upper limit, the signal processing unit deletes the binarized signal of the oldest frame when accumulating the binarized signal of the new frame.

3. The camera device as described in claim 2, wherein, The signal processing unit performs moving average processing or accumulation processing on the detection signal of the specified exclusion frame without performing the binarization processing.

4. The camera device as described in any one of claims 1 to 3, wherein, The signal processing unit multiplies the signal obtained by accumulating the binarized signals of each frame by a predetermined constant.

5. The camera device according to any one of claims 1 to 4, wherein, The signal processing unit determines the number of photons based on a second threshold related to the signal strength that is preset in a way that enables the identification of the number of photons.

6. The camera device according to any one of claims 1 to 5, wherein, The signal processing unit excludes frames with abnormal photon counts from the accumulation object based on a third threshold related to the signal strength that is preset in a way that can identify abnormal photon counts.

Images