Imaging device and imaging method

Abstract

An imaging apparatus 1 comprises: a first image sensor 51 for detecting the luminance of reflected light Lr from a first prism 3 to generate reflected light luminance data Dr; a second image sensor 61 for detecting luminance of transmitted light Lt from the second prism 4 to generate transmitted light luminance data Dt; and a processing unit P for generating reflected light image data Gr and transmitted light image data Gt on the basis of the reflected light luminance data Dr and the transmitted light luminance data Dt that are respectively outputted from the first image sensor 51 and the second image sensor 61 at the same timing, and generating an image dataset Gs in which image data based on reflected light image data Gr and transmitted light image data Gt are combined.

Description

Imaging device and imaging method

[0001] The present disclosure relates to an imaging device and an imaging method.

[0002] An example of a conventional imaging device is an imaging unit described in Patent Document 1. The imaging unit in Patent Document 1 includes an inclined dichroic mirror that separates light from a sample by transmitting or reflecting it according to wavelength, a first imaging unit that captures the light transmitted through the inclined dichroic mirror, and a second imaging unit that captures the light reflected by the inclined dichroic mirror. The first imaging unit and the second imaging unit output the imaging results to an analyzing device. The analyzing device corrects the acquired imaging results in consideration of optical characteristics related to changes in transmittance and reflectance with wavelength in the inclined dichroic mirror. The analyzing device analyzes the corrected image to determine whether the sample is good or bad.

[0003] International Publication No. 2021 / 161684

[0004] In the imaging device described above, even if the image data from the first imaging unit and the second imaging unit are output to the analysis device at the same time, one of the image data may be missing on the analysis device side. In this case, the analysis device may perform a process to fill in the missing image data using image data generated at different times by the first imaging unit and the second imaging unit. However, if image data combining image data generated at different times is used for analysis, it is possible that the accuracy and precision of the analysis may be affected.

[0005] The present disclosure provides an imaging device and an imaging method that can maintain the consistency of timing of combined image data even when image data is missing on the analysis device side.

[0006] The gist of the present disclosure is as follows.

[0007] [1] An imaging device comprising: a first prism having a filter surface whose transmittance and reflectance change monotonically in a predetermined wavelength region, and outputting reflected light reflected by the filter surface from incident light; a second prism optically coupled to the first prism via the filter surface, and outputting transmitted light that is transmitted through the filter surface from the incident light; a first image sensor that detects the luminance of the reflected light from the first prism at a predetermined frame rate and generates reflected light luminance data; a second image sensor that detects the luminance of the transmitted light from the second prism at the predetermined frame rate and generates transmitted light luminance data; and a processing unit that generates first image data and second image data based on the reflected light luminance data and the transmitted light luminance data output from the first image sensor and the second image sensor at the same time, and generates an image data set that combines image data based on the first image data and the second image data.

[0008] In this imaging device, first image data and second image data are generated based on reflected light luminance data and transmitted light luminance data output from the first image sensor and the second image sensor at the same time, and an image dataset is generated by combining these image data. As a result, even if an image dataset is missing at a certain time in an analysis device to which the image dataset is output, the timing of the combined image data is maintained in an image dataset at a different time. Maintaining the timing of the combined image data ensures the accuracy and precision of analysis using the image dataset.

[0009] [2] The imaging device according to [1], wherein the processing unit generates the image data set by combining reflected light image data generated from the reflected light luminance data as the first image data and transmitted light image data generated from the transmitted light luminance data as the second image data, In this case, analysis using the reflected light image data and transmitted light image data included in the image data set can be suitably performed on the analysis device side.

[0010] [3] The imaging device according to [1], wherein the processing unit generates the image data set by combining, as the first image data, luminance image data based on the sum of the reflected light luminance data and the transmitted light luminance data, and, as the second image data, centroid wavelength image data based on relative values ​​for each wavelength region in the reflected light luminance data and the transmitted light luminance data. In this case, analysis using the luminance image data and centroid wavelength image data included in the image data set can be suitably performed on the analysis device side.

[0011] [4] The imaging device according to [1], wherein the processing unit generates the image data set by arranging combined image data from a plurality of image data, including reflected light image data generated from the reflected light luminance data, transmitted light image data generated from the transmitted light luminance data, luminance image data based on the sum of the reflected light luminance data and the transmitted light luminance data, and centroid wavelength image data based on relative values ​​for each wavelength region in the reflected light luminance data and the transmitted light luminance data, in a predetermined direction. In this case, analysis using the image data set can be simplified.

[0012] [5] The imaging device according to any one of [1] to [4], wherein the processing unit generates the image data set including identification information for identifying a boundary of the combined image data. In this case, by referring to the identification information, it becomes easy to distinguish the combined image data, thereby improving the ease of handling of the image data set.

[0013] [6] The imaging device according to any one of [1] to [5], wherein the first image sensor and the second image sensor are the same image sensor. In this case, it is possible to suppress a difference in timing between the data outputs from the first image sensor and the second image sensor, and it is possible to more reliably maintain the consistency of timing of image data included in the image data set.

[0014] [7] An imaging method comprising: a luminance data generation step of detecting, at a predetermined frame rate, the luminance of reflected light reflected from incident light on a filter surface whose transmittance and reflectance change monotonically in a predetermined wavelength range, to generate reflected light luminance data, and detecting, at the predetermined frame rate, the luminance of transmitted light from the incident light that has passed through the filter surface, to generate transmitted light luminance data; an image data generation step of generating first image data and second image data based on the reflected light luminance data and the transmitted light luminance data output at the same time; and an image data set generation step of generating an image data set combining image data based on the first image data and the second image data.

[0015] In this imaging method, first image data and second image data are generated based on reflected light luminance data and transmitted light luminance data output from the first image sensor and the second image sensor at the same time, and an image dataset is generated by combining these image data. As a result, even if an image dataset is missing at a certain time in an analysis device to which the image dataset is output, the timing of the combined image data is maintained in an image dataset at a different time. Maintaining the timing of the combined image data ensures the accuracy and precision of analysis using the image dataset.

[0016] [8] The imaging method according to [7], wherein the image dataset generating step generates the image dataset by combining reflected light image data generated from the reflected light luminance data as the first image data and transmitted light image data generated from the transmitted light luminance data as the second image data. In this case, analysis using the reflected light image data and transmitted light image data included in the image dataset can be suitably performed on the analysis device side.

[0017] [9] The imaging method according to [7], wherein in the image data set generating step, luminance image data based on the sum of the reflected light luminance data and the transmitted light luminance data is combined as the first image data, and centroid wavelength image data based on relative values ​​for each wavelength region in the reflected light luminance data and the transmitted light luminance data is combined as the second image data to generate the image data set. In this case, analysis using the luminance image data and centroid wavelength image data included in the image data set can be suitably performed on the analysis device side.

[0018]

[10] The imaging method according to [7], wherein the image dataset generating step generates the image dataset by arranging combined image data from a plurality of image data including reflected light image data generated from the reflected light luminance data, transmitted light image data generated from the transmitted light luminance data, luminance image data based on the sum of the reflected light luminance data and the transmitted light luminance data, and centroid wavelength image data based on relative values ​​for each wavelength region in the reflected light luminance data and the transmitted light luminance data in a predetermined direction. In this case, analysis using the image dataset can be simplified.

[0019]

[11] The imaging method according to any one of [7] to

[10] , wherein the image dataset generation step generates the image dataset including identification information for identifying a boundary of the combined image data. In this case, by referring to the identification information, it is easy to distinguish the combined image data, thereby improving the ease of handling the image dataset.

[0020]

[12] A first prism including a separation surface that reflects a part of incident light and transmits a part of the incident light, an input surface that inputs a first light reflected at the separation surface, and an output surface that outputs the first light reflected at the input surface; a second prism that is optically coupled to the first prism via the separation surface and includes an output surface that outputs a second light that has transmitted through the separation surface; a filter surface that has a characteristic that transmittance and reflectance change monotonically in a predetermined wavelength region and is formed on any of the output surface of the first prism, the input surface of the first prism, and the output surface of the second prism; and a filter surface that filters the first light, the first transmitted light that has transmitted through the filter surface formed on the output surface of the first prism, or the first transmitted light that has transmitted through the filter surface formed on the input surface of the first prism. a first image sensor that detects, at a predetermined frame rate, a luminance of first reflected light reflected at the filter surface formed on the output surface of the second prism, and outputs first luminance data; a second image sensor that detects, at the predetermined frame rate, a luminance of the second light or a luminance of second transmitted light that has passed through the filter surface formed on the output surface of the second prism, and outputs second luminance data at the same timing as the first luminance data; and a processing unit that generates first image data from the first luminance data, generates second image data from the second luminance data output at the same timing as the first luminance data, and generates an image data set that combines image data based on the first image data and the second image data.

[0021] The imaging device described above provides greater design flexibility by expanding the options for where the filter surface may be formed, while still maintaining the same timing of image data within an image data set, just as with the imaging device described above.

[0022] According to the present disclosure, even if image data is missing on the analysis device side, the timing of the combined image data can be maintained identical.

[0023] 12A is a block diagram showing an imaging system including an imaging device according to an embodiment; FIG. 12B is a diagram showing an example of the appearance of a housing; FIG. 12C is a diagram showing an example of a side view of an imaging device; FIG. 12D is a diagram showing an example of a relationship between incident light and the transmission characteristics of a filter surface; FIG. 12E is a diagram showing an example of a perspective view of an imaging device; FIG. 12F is a diagram showing an example of a rear view of an imaging device; FIG. 12G is a diagram showing another example of a rear view of an imaging device; FIG. 12H is a flowchart showing an example of an imaging method; FIG. 12H is a diagram showing an example of an image dataset; FIG. 12H is a diagram showing another example of an image dataset; FIG. 12H is a diagram showing a case where an analysis device normally acquires image data from an imaging device according to a comparative example; FIG. 12H is a diagram showing a case where one of the image data sets acquired by the analysis device from the imaging device according to the comparative example is missing; FIG. 13A is a diagram showing a case where an analysis device normally acquires image data from an imaging device according to an embodiment; FIG. 13B is a diagram showing a case where one of the image data sets acquired by the analysis device from the imaging device according to an embodiment is missing; FIG. 13H is a diagram showing an example of a side view of an imaging device according to a first modified example; FIG. 13I is a diagram showing an example of a side view of an imaging device according to a second modified example; and FIG. 13I is a diagram showing an example of a side view of an imaging device according to a third modified example.

[0024] Hereinafter, a preferred embodiment of an imaging device according to an embodiment of the present disclosure will be described in detail with reference to the drawings.

[0025] 1 is a block diagram showing an imaging system 100 including an imaging device 1 according to one embodiment. The imaging system 100 includes the imaging device 1 and an analyzing device 20. The imaging device 1 separates incident light L1 (e.g., observation light) from a sample according to wavelength components, generates image data based on luminance data detected for each wavelength component, and outputs the image data to the analyzing device 20. The analyzing device 20 analyzes the image data output from the imaging device 1 to, for example, determine whether the sample is good or bad.

[0026] The sample may be, for example, a light-emitting element such as an LED, mini-LED, μLED, SLD element, laser element, vertical cavity laser element (VCSEL), or OLED, or may be a light-emitting element that adjusts the emission wavelength using a fluorescent material containing nanodots or the like. When the sample is a light-emitting element, the pass / fail judgment is based on, for example, color mottling information between multiple light-emitting elements. The sample may also be a light-reflective semiconductor wafer or food product made of various materials.

[0027] The imaging device 1 includes a prism unit 2 , a first detection board 5 , a second detection board 6 , a first image generation board 7 , a second image generation board 8 , and an image processing board 9 .

[0028] The prism unit 2 includes a first prism 3, a second prism 4, and a filter surface 2a. Incident light L1 incident on the prism unit 2 is separated by the filter surface 2a into reflected light Lr and transmitted light Lt. The reflected light Lr travels through the first prism 3 and is output from the prism unit 2 to a first detection board 5. The transmitted light Lt travels through the second prism 4 and is output from the prism unit 2 to a second detection board 6.

[0029] The first detection substrate 5 includes a first image sensor 51 that receives the reflected light Lr. The first image sensor 51 detects the reflected light Lr from the first prism 3 at a predetermined frame rate. The first image sensor 51 may be an area sensor in which pixels are arranged two-dimensionally, or a line sensor in which pixels are arranged one-dimensionally. If the first image sensor 51 is an area sensor, the first image sensor 51 may be a CCD image sensor, a CMOS image sensor, or an InGaAs image sensor.

[0030] In the first image sensor 51, charges are accumulated in each pixel by exposure to reflected light Lr. The first image sensor 51 outputs an electrical signal (e.g., a signal indicating a voltage value) corresponding to the amount of accumulated charges. Here, the electrical signal is reflected light luminance data Dr based on the magnitude of the luminance of the reflected light Lr received by the first image sensor 51. The first image sensor 51 generates reflected light luminance data Dr for each frame time corresponding to a predetermined frame rate. The first image sensor 51 outputs the reflected light luminance data Dr for each pixel to a first image generator 71 included in the first image generation board 7.

[0031] The second detection substrate 6 includes a second image sensor 61 that receives the transmitted light Lt. The second image sensor 61 detects the transmitted light Lt from the second prism 4 at a predetermined frame rate. This predetermined frame rate is the same as the frame rate at which the first image sensor 51 detects. The second image sensor 61 may be an area sensor in which pixels are arranged two-dimensionally, or may be a line sensor in which pixels are arranged one-dimensionally. If the second image sensor 61 is an area sensor, the second image sensor 61 may be a CCD image sensor, a CMOS image sensor, or an InGaAs image sensor.

[0032] In the second image sensor 61, charges are accumulated in each pixel by exposure to transmitted light Lt. The second image sensor 61 outputs an electrical signal (e.g., a signal indicating a voltage value) corresponding to the amount of accumulated charges. Here, the electrical signal is transmitted light luminance data Dt based on the magnitude of the luminance of the transmitted light Lt received by the second image sensor 61. The second image sensor 61 generates transmitted light luminance data Dt for each frame time corresponding to a predetermined frame rate. The second image sensor 61 outputs the transmitted light luminance data Dt for each pixel to a second image generator 81 included in the second image generation board 8.

[0033] The timing at which the reflected light luminance data Dr is output by the first image sensor 51 is the same as the timing at which the transmitted light luminance data Dt is output by the second image sensor 61. For example, the first image sensor 51 and the second image sensor 61 may be the same image sensor. In this case, the frame time of the first image sensor 51 may be synchronized with the frame time of the second image sensor 61. When the frame time is composed of an exposure time, a luminance data readout time, and a luminance data transfer time, it is sufficient that at least one of these times is synchronized between the first image sensor 51 and the second image sensor 61. For example, the exposure time may be synchronized between the first image sensor 51 and the second image sensor 61.

[0034] The first image generation board 7 includes a first image generator 71. The first image generator 71 generates first image data based on the reflected light luminance data Dr. The first image sensor 51 and the first image generator 71 are electrically connected via, for example, a flexible flat cable. The first image generator 71 generates reflected light image data Gr from the reflected light luminance data Dr. The reflected light image data Gr is an example of first image data. The first image generator 71 may, for example, generate the reflected light image data Gr by integrating multiple acquired pieces of reflected light luminance data Dr. The first image generator 71 may also generate the reflected light image data Gr by correcting the linearity of the reflected light luminance data Dr. The first image generator 71 outputs the generated reflected light image data Gr to an image processor 91 included in the image processing board 9. The first image generator 71 may be configured using an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), for example.

[0035] The second image generation board 8 includes a second image generator 81. The second image generator 81 generates second image data based on the transmitted light luminance data Dt. The second image sensor 61 and the second image generator 81 are electrically connected via, for example, a flexible flat cable. The second image generator 81 generates transmitted light image data Gt from the transmitted light luminance data Dt. The transmitted light image data Gt is an example of second image data. The second image generator 81 may generate the transmitted light image data Gt by, for example, integrating multiple acquired transmitted light luminance data Dt. The second image generator 81 may also generate the transmitted light image data Gt by correcting the linearity of the transmitted light luminance data Dt. The second image generator 81 outputs the generated transmitted light image data Gt to an image processor 91 included in the image processing board 9. The second image generator 81 may be configured using an FPGA, an ASIC, or the like.

[0036] The first image data or the second image data includes not only the reflected light image data Gr and the transmitted light image data Gt, but also various variations such as luminance image data or centroid wavelength data as calculated image data processed based on the reflected light luminance data Dr and the transmitted light luminance data Dt. The luminance image data and centroid wavelength data will be described later. In the following description, as an example, the first image data will be described as the reflected light image data Gr and the second image data will be described as the transmitted light image data Gt.

[0037] The timing at which the reflected light image data Gr is output by the first image generator 71 is the same as the timing at which the transmitted light image data Gt is output by the second image generator 81. For example, the image processor 91 included in the image processing board 9 may transmit a synchronization signal to the first image generator 71 and the second image generator 81, thereby synchronizing the output timing of the first image generator 71 and the output timing of the second image generator 81.

[0038] The image processor 91 generates an image data set Gs by combining image data based on the first image data and the second image data. The image processor 91, for example, combines reflected light image data Gr and transmitted light image data Gt and outputs the combined image data Gs to the outside. The image processor 91 generates an image data set Gs by combining image data based on the reflected light image data Gr and transmitted light image data Gt. The image data set Gs will be described later. The image processor 91 then outputs the image data set Gs to an analysis device 20 provided outside the imaging device 1. The image processor 91 may be configured with an FPGA, an ASIC, or the like. The image processor 91 communicates with the first image generator 71 and the second image generator 81, for example, via serial transmission or parallel transmission.

[0039] The above-described first image generator 71, second image generator 81, and image processor 91 constitute a processing unit P in the imaging device 1. That is, the processing unit P generates reflected light image data Gr and transmitted light image data Gt based on the reflected light luminance data Dr and transmitted light luminance data Dt output at the same time from the first image sensor 51 and the second image sensor 61. The processing unit P then generates an image data set Gs by combining image data based on the reflected light image data Gr and transmitted light image data Gt.

[0040] The first detection substrate 5 may further include a Peltier element (not shown). The Peltier element is a thermoelectric cooling element that utilizes the Peltier effect. The Peltier element cools the first image sensor 51. The first image generation substrate 7 may further include a drive element for driving the Peltier element. The drive element is, for example, a current source. When the drive element passes a current through the Peltier element, the Peltier element cools the first image sensor 51 to a temperature that corresponds to the magnitude of the current. Similarly, the second detection substrate 6 may further include a Peltier element for cooling the second image sensor 61, and the second image generation substrate 8 may further include a drive element for driving the Peltier element.

[0041] The imaging device 1 may further include a motor driver (not shown). The first detection board 5 and the second detection board 6 may each be placed on a support table (not shown) that is movable in two or three dimensions, for example. The support table may be motor-driven and controlled by the motor driver. Furthermore, the position of the support table may be controlled by, for example, the first image generator 71 and the second image generator 81. When changing the positions of the first detection board 5 and the second detection board 6, the first image generator 71 and the second image generator 81 may drive the motor driver and control the support table so that the first detection board 5 and the second detection board 6 reach desired positions.

[0042] [Specific Structure of Imaging Device] Next, a specific structure of the imaging device 1 will be described. The imaging device 1 further includes a housing 10 that houses the prism unit 2, the first detection board 5, the second detection board 6, the first image generation board 7, the second image generation board 8, and the image processing board 9. FIG. 2 is a diagram showing an example of the appearance of the housing 10. The housing 10 is, for example, a hexahedral box-like cubic shape. The housing 10 is made of, for example, metal, and protects and shields the functional components housed therein from light.

[0043] A mount 11 is provided on one side of the housing 10. The mount 11 is a connection portion for attaching a lens to the housing 10. By attaching the lens to the housing 10 via the mount 11, the lens and the prism unit 2 are optically connected. The lens is arranged, for example, so that incident light L1 from the sample is focused onto each of the first image sensor 51 and the second image sensor 61. The mount 11 may be, for example, a screw mount such as an M52 mount or a C mount, or a bayonet mount such as an F mount.

[0044] A connector C1 for electrically connecting the analysis device 20 and the imaging device 1 may be formed on another side surface of the housing 10. The imaging device 1 and the analysis device 20 are connected by, for example, a Camera Link interface, and the imaging device 1 performs high-speed data transfer of image data to the analysis device 20 via this interface. The connector C1 may be, for example, an SDR (Shrunk Delta Ribbon connector) connector or an MDR (Mini Delta Ribbon) connector.

[0045] Next, the configuration of the prism unit 2 will be described. FIG. 3 is a diagram schematically illustrating an example of a side view of the imaging device 1. The first prism 3 has a separation surface 3a that separates the incident light L1 incident from the mount 11. In this embodiment, the separation surface 3a is formed with a filter surface 2a having characteristics in which the transmittance and reflectance change monotonically over a predetermined wavelength range. Here, the filter surface 2a is formed with a light dividing element (beam splitter) called a linear reflectance gradient (LRG) dichroic mirror. The inclined dichroic mirror is formed, for example, with a dielectric multilayer film, and transmits and reflects the incident light L1 according to its wavelength, thereby dividing the incident light L1 into transmitted light Lt and reflected light Lr.

[0046] 4 is a diagram illustrating an example of the relationship between incident light L1 and the transmission characteristics of the filter surface 2a. For example, when a sample includes μLEDs and the incident light L1 has a center wavelength λ 2 different from the center wavelength λ 2 , １ ~λ ３ As shown in Fig. 4, the wavelength range in which the transmittance of the filter surface 2a changes linearly is set based on the wavelength range of the incident light L1. In the example of Fig. 4, the transmittance of the filter surface 2a changes linearly at a center wavelength λ １ In the wavelength region smaller than λ, it is close to 0%. １ ~λ ３ The transmittance of the filter surface 2a increases at a constant rate in the wavelength region including the center wavelength λ ３In the wavelength range greater than 100%, the transmittance approaches 100%. The transmittance and reflectance of the filter surface 2a are negatively correlated, so that as one increases, the other decreases. The wavelength range in which the transmittance of the filter surface 2a increases at a constant rate is, for example, 400 nm to 800 nm.

[0047] The first prism 3 further has an input surface 3b and an output surface 3c. The input surface 3b faces the inner wall of the side of the housing 10 on which the mount 11 is provided. The first prism 3 is optically connected to the first image sensor 51 included in the first detection board 5. The reflected light Lr reflected at the separation surface 3a is further reflected at the input surface 3b. The input surface 3b serves as an input surface for the incident light L1, but serves as a total reflection surface for the reflected light Lr reflected at the separation surface 3a. The reflected light Lr then travels within the first prism 3, is output from the output surface 3c, and enters the first image sensor 51.

[0048] The second prism 4 is optically coupled to the first prism 3 via the separation surface 3a. The second prism 4 has an input surface 4a and an output surface 4b. The input surface 4a is bonded to the separation surface 3a without any gap. This optically couples the first prism 3 and the second prism 4. The second prism 4 is optically connected to a second image sensor 61 included in the second detection substrate 6. The transmitted light Lt that has passed through the separation surface 3a travels within the second prism 4, is output from the output surface 4b, and input to the second image sensor 61.

[0049] Next, the internal structure of the housing 10 will be described in more detail with reference to Figures 3, 5, and 6. Figure 5 is a diagram schematically showing an example of a perspective view of the imaging device 1, and Figure 6 is a diagram schematically showing an example of a rear view of the imaging device 1.

[0050] 3, the first image sensor 51 is mounted on the surface of the first detection substrate 5. The first image sensor 51 is disposed so as to face the first prism 3. The first image sensor 51 may be disposed with a space therebetween, for example, by a spacer, or may be disposed so as to be in direct contact with the output surface 3c of the first prism 3. The Peltier element may be mounted on the back surface of the first detection substrate 5 opposite the surface on which the first image sensor 51 is mounted. In addition, a heat sink (not shown) may be connected to the Peltier element to dissipate heat generated by the first image sensor 51.

[0051] The second image sensor 61 is mounted on the surface of the second detection substrate 6. The second image sensor 61 is disposed so as to face the second prism 4. The second image sensor 61 may be disposed with a space between it and the output surface 4b of the second prism 4, for example, by a spacer, or may be disposed so as to be in direct contact with the output surface 4b. As with the first detection substrate 5, a Peltier element (not shown) and a heat sink (not shown) may be mounted on the back surface of the second detection substrate 6.

[0052] The imaging device 1 further includes a holding member 12 within the housing 10. The holding member 12 integrally holds the first prism 3, the second prism 4, the first detection board 5, and the second detection board 6. As shown in FIG. 5 , the holding member 12 may include a pair of legs 121 and 122. The pair of legs 121 and 122 are arranged to sandwich the first prism 3 and the second prism 4. Ends of the pair of legs 121 and 122 on the side where the incident light L1 is incident are coupled to the mount 11. A heat sink is fixed to the end of the pair of legs 121 and 122 opposite the end coupled to the mount 11 via a spacer that passes through a hole in the second detection board 6. A Peltier element is sandwiched between the heat sink and the second detection board 6. The upper surfaces 121 a and 122 a of the pair of legs 121 and 122 are notched in a V-shape in side view. A heat sink is fixed to the inclined portion formed by this shape via a spacer that passes through a hole in the first detection board 5. A Peltier element is sandwiched between this heat sink and the first detection board 5.

[0053] The first detection board 5 further includes a first terminal 53. The first terminal 53 is an output terminal for reflected light luminance data Dr from the first image sensor 51. The second detection board 6 further includes a second terminal 63. The second terminal 63 is an output terminal for transmitted light luminance data Dt from the second image sensor 61. The first terminal 53 and the second terminal 63 are, for example, FFC connectors that mate with flexible flat cables.

[0054] The first detection board 5 and the second detection board 6 are held by the prisms (the first prism 3 and the second prism 4) via image sensors (the first image sensor 51 and the second image sensor 61) so as to protrude in opposite directions from the holding member 12. As shown in FIGS. 5 and 6 , the first detection board 5 includes a protruding portion 54 that protrudes beyond the leg portions 121 of the holding member 12 on the side opposite the first prism 3. The second detection board 6 includes a protruding portion 64 that protrudes beyond the leg portions 122 of the holding member 12 on the side opposite the second prism 4. The protruding portions 54 and 64 are located in opposite directions in a direction perpendicular to the traveling direction of the incident light L1. The first terminal 53 is located on the protruding portion 54. The second terminal 63 is located on the protruding portion 64.

[0055] In this embodiment, the arrangement positions of the first image sensor 51 and the first terminal 53 on the first detection board 5 are the same as the arrangement positions of the second image sensor 61 and the second terminal 63 on the second detection board 6. The first detection board 5 has a shape whose longitudinal direction is a direction perpendicular to the traveling direction of the incident light L1. The arrangement direction of the first image sensor 51 and the first terminal 53 on the first detection board 5 coincides with the longitudinal direction of the first detection board 5. The second detection board 6 has a shape whose longitudinal direction is a direction parallel to the longitudinal direction of the first detection board 5. The arrangement direction of the second image sensor 61 and the second terminal 63 on the second detection board 6 coincides with the longitudinal direction of the second detection board 6.

[0056] The first detection board 5 and the second detection board 6 are held by the holding member 12 in a state in which one of them is inverted relative to the other in the in-plane direction of the board. In the example of Figures 5 and 6, the first detection board 5 is arranged in a state rotated 180 degrees about the rotation axis relative to the second detection board 6, when the direction perpendicular to the board is taken as the rotation axis.

[0057] As shown in Fig. 5, when an imaginary line A1 is drawn from the mount 11 along the upper surfaces 121a, 122a of the holding member 12 to the end of the holding member 12 connected to the second detection board 6, the first image sensor 51 and the second image sensor 61 are both located on the imaginary line A1. As shown in Fig. 6, for example, when the upper surfaces 121a, 122a of the legs 121, 122 of the holding member 12 are viewed from above, the first image sensor 51 and the second image sensor 61 are located in a straight line on the imaginary line A1. As a result, the first terminal 53 and the second terminal 63 are arranged symmetrically with respect to the imaginary line A1.

[0058] As shown in FIG. 3 , the imaging device 1 may further include a cut filter 13 and a depolarizer 14. In the example of FIG. 3 , the cut filter 13 and the depolarizer 14 are disposed before the first prism 3. The incident light L1 passes through the cut filter 13 and the depolarizer 14 before reaching the separation surface 3a. The cut filter 13 is, for example, a bandpass filter. The cut filter 13 may pass light within a wavelength range in which the transmittance of the filter surface 2a on the separation surface 3a increases at a constant rate, and block light within other wavelength ranges. The depolarizer 14 converts the incident light L1 into unpolarized light when the incident light L1 is p-polarized, s-polarized, or elliptically polarized. The transmittance of the incident light L1 on the separation surface 3a may vary between p-polarized and s-polarized light. The depolarizer 14 can eliminate the polarization dependency of the incident light L1.

[0059] Next, the internal structure of the housing 10 of the imaging device 1, including the first image generation board 7, the second image generation board 8, and the image processing board 9, will be described. FIG. 7 is a diagram schematically illustrating another example of a rear view of the imaging device 1. FIG. 7 shows the state in which the rear panel of the housing 10 has been removed to reveal the interior. The housing 10 has inner walls 10a and 10b that face each other. As shown in FIG. 7, the first image generation board 7 is disposed on the inner wall 10a. The second image generation board 8 is disposed on the inner wall 10b. The in-plane direction of the first image generation board 7 is parallel to the inner wall 10a, and the in-plane direction of the second image generation board 8 is parallel to the inner wall 10b. In other words, the first image generation board 7 and the second image generation board 8 are disposed so as to face each other.

[0060] The first image generating board 7 faces the protruding portion 54 of the first detection board 5 that protrudes from the holding member 12. The second image generating board 8 faces the protruding portion 64 of the second detection board 6 that protrudes from the holding member 12. The opposing direction of the first image generating board 7 and the protruding portion 54 of the first detection board 5 is along the arrangement direction of the first image sensors 51 and the first terminals 53 on the first detection board 5. In the example of FIG. 7 , the longitudinal direction of the first detection board 5 and the opposing direction of the first image generating board 7 and the protruding portion 54 of the first detection board 5 coincide with each other. The first image generating board 7 is located on an extension of the longitudinal direction of the first detection board 5. The opposing direction of the second image generating board 8 and the protruding portion 64 of the second detection board 6 is along the arrangement direction of the second image sensors 61 and the second terminals 63 on the second detection board 6. 7, the longitudinal direction of the second detection board 6 coincides with the opposing direction of the second image generation board 8 and the protruding portion 64 of the second detection board 6. The second image generation board 8 is located on an extension of the longitudinal direction of the second detection board 6.

[0061] The first image generation board 7 further includes a third terminal 73. The third terminal 73 is an input terminal for the reflected light luminance data Dr output from the first terminal 53. The second image generation board 8 further includes a fourth terminal 83. The fourth terminal 83 is an input terminal for the transmitted light luminance data Dt output from the second terminal 63. The third terminal 73 and the fourth terminal 83 are, for example, FFC connectors that mate with flexible flat cables (FFC). The third terminal 73 and the first terminal 53 are connected by a flexible flat cable FC1. The flexible flat cable FC1 is wired from the first terminal 53 perpendicular to the board, bent parallel to the board in the direction opposite to the third terminal 73, and then folded back toward the third terminal 73 to connect to the third terminal 73. The fourth terminal 83 and the second terminal 63 are connected by a flexible flat cable FC2. The flexible flat cable FC2 is wired from the second terminal 63 perpendicular to the substrate, bent parallel to the substrate in the direction opposite to the fourth terminal 83, and then folded back toward the fourth terminal 83 to be connected to the fourth terminal 83. Note that an FPC (Flexible Printed Circuits) may be used instead of the flexible flat cable.

[0062] The housing 10 further includes an inner wall 10c connecting the inner wall 10a and the inner wall 10b. In this embodiment, the housing 10 has a cubic shape, and the inner wall 10c is perpendicular to the inner walls 10a and 10b. The image processing board 9 is disposed on the inner wall 10c. As a result, the image processing board 9 is disposed in the housing 10 so as not to face either the protruding portion 54 of the first detection board 5 from the holding member 12 or the protruding portion 64 of the second detection board 6 from the holding member 12, but is perpendicular to both the opposing direction of the first image generation board 7 and the protruding portion 54 and the opposing direction of the second image generation board 8 and the protruding portion 64.

[0063] [Imaging Method] Next, a description will be given of an imaging method using the imaging device 1. Fig. 8 is a flowchart showing an example of an imaging method (hereinafter, method MT1). In method MT1, steps ST1 to ST4 can be repeatedly performed in this order.

[0064] First, the first image sensor 51 detects the luminance of the reflected light Lr to generate reflected light luminance data Dr, and the second image sensor 61 detects the luminance of the transmitted light Lt to generate transmitted light luminance data Dt (step ST1: luminance data generation step). In step ST1, the first image sensor 51 and the second image sensor 61 output the reflected light luminance data Dr and the transmitted light luminance data Dt at the same time.

[0065] Next, the first image generator 71 generates reflected light image data Gr by combining the reflected light luminance data Dr for each predetermined wavelength region, and the second image generator 81 generates transmitted light image data Gt by combining the transmitted light luminance data Dt for each predetermined wavelength region (step ST2: image data generation step). In step ST2, the first image generator 71 and the second image generator 81 output the reflected light image data Gr and the transmitted light image data Gt at the same time.

[0066] In step ST2, one of the first image generator 71 and the second image generator 81 may invert the orientation of the image in the image data to be output. As described above, the first detection board 5 and the second detection board 6 are held by the holding member 12 with one of them inverted relative to the other in the in-plane direction of the board. Therefore, the orientation of the reflected light image data Gr output from the first image generator 71 and the orientation of the transmitted light image data Gt output from the second image generator 81 may be inverted. In this case, by having one of the first image generator 71 and the second image generator 81 invert the orientation of the image in the image data to be output, the reflected light image data Gr and the transmitted light image data Gt can be output in the same orientation.

[0067] The image orientation inversion in the image data as described above may be performed already when reading out the luminance data from the first image sensor 51 and the second image sensor 61. For example, if the first image sensor 51 and the second image sensor 61 are rolling shutter type, the order in which luminance data is read out from one image sensor may be reversed from the order in which luminance data is read out from the other image sensor. For example, when reading out pixel rows, one image sensor may read out rows one by one from the first row to the last row, while the other image sensor may read out rows one by one from the last row to the first row. The first image sensor 51 and the second image sensor 61 may be global shutter type.

[0068] Next, the image processor 91 generates an image data set Gs (step ST3: image data set generation step). As described above, the first image data or the second image data includes not only the reflected light image data Gr and the transmitted light image data Gt, but also various variations of luminance image data or centroid wavelength data as calculated image data processed based on the reflected light luminance data Dr and the transmitted light luminance data Dt. The image processor 91 generates an image data set Gs consisting of at least two pieces of image data by combining one each of the reflected light image data Gr, transmitted light image data Gt, and calculated image data.

[0069] The image processor 91 may generate luminance image data based on the sum of the reflected light luminance data Dr and the transmitted light luminance data Dt as the calculated image data. The luminance image data is data indicating the total amount of light, which is the sum of the luminance of the transmitted light Lt and the luminance of the reflected light Lr. The luminance image data may be generated as first image data or second image data. As shown in FIG. 4 , the pixel value of each pixel included in the luminance image data is expressed as T+R, where R is the count value of the luminance of the reflected light Lr received by each pixel of the first image sensor 51 and T is the count value of the luminance of the transmitted light Lt received by each pixel of the second image sensor 61.

[0070] The image processor 91 may generate, as the calculated image data, centroid wavelength image data that takes relative values ​​for each wavelength region in the reflected light luminance data Dr and the transmitted light luminance data Dt. The centroid wavelength image data is data that indicates the relative values ​​between the luminance of the transmitted light Lt and the luminance of the reflected light Lr. The centroid wavelength image data may be generated as first image data or second image data. As shown in FIG. 4 , the pixel value of each pixel included in the centroid wavelength image data is expressed as (T−R) / (2×(T+R)), where R is the count value of the luminance of the reflected light Lr received by each pixel of the first image sensor 51 and T is the count value of the luminance of the transmitted light Lt received by each pixel of the second image sensor 61.

[0071] The image processor 91 may generate, as the calculated image data, differential image data that is the difference between the reflected light luminance data Dr and the transmitted light luminance data Dt. Alternatively, the image processor 91 may generate, as the calculated image data, data that is easy to use in the analysis device 20. For example, the image processor 91 may perform filter processing on the reflected light image data Gr and the transmitted light image data Gt.

[0072] The image processor 91 generates the image data set Gs by combining any of the image data described above. For example, the image processor 91 generates the image data set Gs by combining reflected light image data Gr generated from reflected light luminance data Dr and transmitted light image data Gt generated from transmitted light luminance data Dt. For example, the image processor 91 generates the image data set Gs by combining luminance image data and centroid wavelength image data. The image data set Gs does not need to consist of two pieces of image data; the image processor 91 may also generate the image data set Gs by combining three or more images.

[0073] The image processor 91 generates an image data set Gs by arranging a combination of a plurality of image data, including reflected light image data Gr, transmitted light image data Gt, luminance image data, and centroid wavelength image data, in a predetermined direction. In the following description, an example will be described in which the image processor 91 generates the image data set Gs by combining the reflected light image data Gr and transmitted light image data Gt.

[0074] 9 to 11 are diagrams showing an example of the image data set Gs. The predetermined direction in which the image processor 91 arranges the reflected light image data Gr and the transmitted light image data Gt will be described using a three-dimensional coordinate system as an example. In the three-dimensional coordinate system shown in FIGS. 9 to 11, the direction in which the pixels of the image data are arranged horizontally is defined as the X direction, the direction in which the pixels of the image data are arranged vertically is defined as the Z direction, and the direction perpendicular to the X and Z directions is defined as the Y direction.

[0075] As shown in FIG. 9 , the image processor 91 may arrange the reflected light image data Gr and the transmitted light image data Gt in the X direction. As shown in FIG. 10 , the image processor 91 may arrange the reflected light image data Gr and the transmitted light image data Gt in the Z direction. As shown in FIG. 11 , the image processor 91 may arrange the reflected light image data Gr and the transmitted light image data Gt in the Y direction. In the example of FIG. 11 , the image processor 91 arranges the reflected light image data Gr and the transmitted light image data Gt so that their image planes overlap. In addition to the above example, the image processor 91 may arrange the reflected light image data Gr and the transmitted light image data Gt in any manner in the three-dimensional coordinate system. For example, the image processor 91 may arrange the reflected light image data Gr and the transmitted light image data Gt diagonally on a plane defined by the X and Z directions.

[0076] The image processor 91 generates the image data set Gs including identification information R for identifying boundaries of the combined image data. As shown in Figures 9 and 10, the image processor 91 may embed a string of dummy pixels with no pixel values ​​as the identification information R between the reflected light image data Gr and the transmitted light image data Gt in the X or Z direction. Alternatively, as shown in Figure 11, the image processor 91 may embed a dummy image as the identification information R between the reflected light image data Gr and the transmitted light image data Gt in the Y direction. Alternatively, the image processor 91 may generate data indicating image alignment information (for example, in the case of 4-bit serial data, "0001" is used when the image data are aligned in the X direction) as the identification information R along with the image data set Gs.

[0077] Finally, the image processor 91 outputs the image data set Gs to the analysis device 20 (step ST4: image data set output step). Since the timing of the image data is maintained uniform within the image data set Gs, the analysis device 20 can analyze the image data without adjusting the timing between the image data, for example.

[0078] [Operation and Effect] In the analysis device according to the comparative example, as shown in FIG. 12( a), reflected light image data Gr generated from reflected light luminance data Dr acquired by the first image sensor 51 and transmitted light image data Gt generated from transmitted light luminance data Dt acquired by the second image sensor 61 are each output independently to the analysis device 20. The analysis device 20 analyzes a sample using the reflected light image data Gr and transmitted light image data Gt output at the same time. However, when the analysis device 20 receives the reflected light image data Gr and transmitted light image data Gt, one of the reflected light image data Gr and transmitted light image data Gt output at the same time may be missing. In this case, as shown in FIG. 12( b), the missing transmitted light image data Gt may be complemented by transmitted light image data Gt acquired at the next time in the analysis device 20. In this case, because image data generated at different times by the first image sensor 51 and the second image sensor 61 are combined, the timing of the combined image data may not be consistent, which may affect the analysis results of the sample.

[0079] In contrast, in the imaging device 1 according to this embodiment, as shown in FIG. 13A , an image data set Gs is output to the analysis device 20, which combines reflected light image data Gr generated from reflected light luminance data Dr acquired by the first image sensor 51 and transmitted light image data Gt generated from transmitted light luminance data Dt acquired by the second image sensor 61. The analysis device 20 outputs the image data set Gs, allowing analysis of the sample using the reflected light image data Gr and transmitted light image data Gt output at the same time. However, the image data set Gs received at a certain time may be missing. Even in this case, as shown in FIG. 13B , the image data set Gs received at the next time by the analysis device 20 maintains the same timing of the combined image data. This ensures the accuracy and precision of analysis using the image data set Gs.

[0080] In this embodiment, in the image dataset generation step (step ST4), the processing unit P combines the reflected light image data Gr generated from the reflected light luminance data Dr and the transmitted light image data Gt generated from the transmitted light luminance data Dt to generate the image dataset Gs. In this case, analysis using the reflected light image data Gr and transmitted light image data Gt included in the image dataset Gs can be suitably performed on the analysis device 20 side.

[0081] In this embodiment, in the image dataset generation step (step ST4), the processing unit P generates, as calculated image data, luminance image data based on the sum of the reflected light luminance data Dr and the transmitted light luminance data Dt, and centroid wavelength image data based on relative values ​​for each wavelength region in the reflected light luminance data Dr and the transmitted light luminance data Dt, and combines the luminance image data and centroid wavelength image data to generate the image dataset Gs. In this case, analysis using the luminance image data and centroid wavelength image data included in the image dataset Gs can be suitably performed on the analysis device 20 side.

[0082] In this embodiment, in the image dataset generation step (step ST4), the processing unit P generates the image dataset Gs by arranging a combination of multiple image data sets, including the reflected light image data Gr, the transmitted light image data Gt, the luminance image data, and the centroid wavelength image data, in a predetermined direction, thereby simplifying the analysis using the image dataset Gs.

[0083] In this embodiment, in the image dataset generation step (step ST4), the processing unit P generates the image dataset Gs including identification information R for identifying the boundary of the combined image data. In this case, by referring to the identification information R, it becomes easy to distinguish the combined image data, thereby improving the ease of handling of the image dataset Gs.

[0084] In the imaging device 1 according to this embodiment, the first image sensor 51 and the second image sensor 61 are the same image sensor. In this case, it is possible to suppress a difference in timing between the data outputs from the first image sensor 51 and the second image sensor 61, and it is possible to more reliably maintain the consistency of timing of the image data included in the image data set Gs.

[0085] [Modifications] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments.

[0086] Instead of the separation surface 3a, the filter surface 2a may be formed on any of the output surface 3c of the first prism 3, the input surface 3b of the first prism 3, and the output surface 4b of the second prism 4. In this case, the separation surface 3a may be formed by an optical element without wavelength selectivity. For example, the separation surface 3a may be formed by a beam splitter that separates the incident light L1 at a specific intensity ratio regardless of wavelength. The intensity ratio may be 1:1 or any other ratio. In the following description, a portion of the incident light L1 reflected by the separation surface 3a is referred to as the first incident light L11 (first light), and a portion of the incident light L1 transmitted through the separation surface 3a is referred to as the second incident light L12 (second light). An optical element (filter surface 2a) having a characteristic that its transmittance and reflectance change monotonically in a predetermined wavelength range is disposed in either the optical path of the first incident light L11 or the optical path of the second incident light L12.

[0087] [First Modification] FIG. 14 is a diagram showing an example of a side view of an imaging device 1A according to a first modification. In the imaging device 1A, a filter surface 2a is formed on the output surface 3c of the first prism 3. First, incident light L1 is separated into first incident light L11 and second incident light L12 at the separation surface 3a. The first incident light L11 is reflected at the input surface 3b of the first prism 3 and then split into transmitted light Lt and reflected light Lr (not shown) at the output surface 3c. Of the split light, the first image sensor 51 receives only the transmitted light Lt (first transmitted light). Meanwhile, the second incident light L12 travels through the second prism 4 and is input to the second image sensor 61. The first image sensor 51 outputs transmitted light luminance data Dt (first luminance data in the first modification) based on the luminance of the transmitted light Lt to the first image generator 71.

[0088] The second image sensor 61 outputs luminance data (second luminance data in the first modified example) based on the magnitude of the luminance of the second incident light L12 to the second image generator 81. The first image generator 71 generates transmitted light image data Gt (first image data in the first modified example) based on the transmitted light luminance data Dt. The second image generator 81 generates image data (second image data in the first modified example) based on the luminance data of the second incident light L12. Here, the reflected light Lr split at the output surface 3c is represented by the difference between the second incident light L12 and the transmitted light Lt. In other words, the image processor 91 may generate the reflected light image data Gr by calculating the difference between the image data of the second incident light L12 and the transmitted light image data Gt before generating the image data set Gs.

[0089] [Second Modification] FIG. 15 is a diagram showing an example of a side view of an imaging device 1B according to a second modification. In the imaging device 1B, a filter surface 2a is formed on the input surface 3b of the first prism 3. First, the incident light L1 is separated into a first incident light L11 and a second incident light L12 at the separation surface 3a. The first incident light L11 is split into a transmitted light Lt (not shown) and a reflected light Lr at the input surface 3b of the first prism 3. Of the split light, the first image sensor 51 receives only the reflected light Lr (first reflected light). Meanwhile, the second incident light L12 travels through the second prism 4 and is input to the second image sensor 61. The first image sensor 51 outputs reflected light luminance data Dr (first luminance data in the second modification) based on the luminance of the reflected light Lr to the first image generator 71.

[0090] The second image sensor 61 outputs luminance data (second luminance data in the second modified example) based on the magnitude of the luminance of the second incident light L12 to the second image generator 81. The first image generator 71 generates reflected light image data Gr (first image data in the second modified example) based on the reflected light luminance data Dr. The second image generator 81 generates image data (second image data in the second modified example) based on the luminance data of the second incident light L12. Here, the transmitted light Lt split at the input surface 3b is represented by the difference between the second incident light L12 and the reflected light Lr. That is, the image processor 91 may generate the transmitted light image data Gt by calculating the difference between the image data of the second incident light L12 and the reflected light image data Gr before generating the image data set Gs.

[0091] [Third Modification] Figure 16 is a diagram showing an example of a side view of an imaging device 1C according to a third modification. In the imaging device 1C, a filter surface 2a is formed on the output surface 4b of the second prism 4. First, the incident light L1 is separated into a first incident light L11 and a second incident light L12 at the separation surface 3a. The second incident light L12 is split into a transmitted light Lt and a reflected light Lr (not shown) at the output surface 4b of the second prism 4. Of the split light, the second image sensor 61 receives only the transmitted light Lt (second transmitted light). Meanwhile, the first incident light L11 travels through the first prism 3 and enters the first image sensor 51.

[0092] The second image sensor 61 outputs transmitted light luminance data Dt (second luminance data in the third modified example) based on the luminance of the transmitted light Lt to the second image generator 81. The first image sensor 51 outputs luminance data (first luminance data in the third modified example) based on the luminance of the first incident light L11 to the first image generator 71. The second image generator 81 generates transmitted light image data Gt (second luminance data in the third modified example) based on the transmitted light luminance data Dt. The first image generator 71 generates image data (first image data in the third modified example) based on the luminance data of the first incident light L11. Here, the reflected light Lr split at the output surface 4b is represented by the difference between the first incident light L11 and the transmitted light Lt. That is, the image processor 91 may generate reflected light image data Gr by calculating the difference between the image data of the first incident light L11 and the transmitted light image data Gt before generating the image data set Gs.

[0093] Even with configurations such as those of the first to third modified examples, centroid wavelength image data, luminance image data, etc. can be generated from the image data of the transmitted light Lt and the reflected light Lr, and various analyses can be performed using the centroid wavelength image data, luminance image data, etc. Furthermore, with the imaging devices 1A to 1C, the options for the location where the filter surface 2a is formed are expanded, improving the degree of freedom in design. Furthermore, as with the imaging device 1, the consistency of the timing of the image data within the image data set Gs can be maintained.

[0094] 1, 1A, 1B, 1C...imaging device, 2a...filter surface, 3...first prism, 3a...separation surface, 3b...input surface of first prism, 3c...output surface of first prism, 4...second prism, 4b...output surface of second prism, 51...first image sensor, 61...second image sensor, Dr...reflected light luminance data, Dt...transmitted light luminance data, Gr...reflected light image data (first image data), Gs...image data set, Gt...transmitted light image data (second image data), L1...incident light, L11...first incident light (first light), L12...second incident light (second light), Lr...reflected light, Lt...transmitted light, P...processing unit, R...identification information.

Claims

1. An imaging device comprising: a first prism having a filter surface whose transmittance and reflectance change monotonically in a predetermined wavelength range, and which outputs reflected light reflected by said filter surface from incident light; a second prism optically coupled to said first prism via said filter surface, and which outputs transmitted light that is transmitted through said filter surface from said incident light; a first image sensor that detects the luminance of the reflected light from said first prism at a predetermined frame rate and generates reflected light luminance data; a second image sensor that detects the luminance of the transmitted light from said second prism at the same frame rate and generates transmitted light luminance data; and a processing unit that generates first image data and second image data based on the reflected light luminance data and the transmitted light luminance data output from said first image sensor and said second image sensor at the same time, and generates an image data set that combines image data based on said first image data and said second image data.

2. The imaging device of claim 1, wherein the processing unit generates the image data set by combining reflected light image data generated from the reflected light luminance data as the first image data and transmitted light image data generated from the transmitted light luminance data as the second image data.

3. The imaging device of claim 1, wherein the processing unit generates the image data set by combining luminance image data based on the sum of the reflected light luminance data and the transmitted light luminance data as the first image data, and centroid wavelength image data based on relative values ​​for each wavelength region in the reflected light luminance data and the transmitted light luminance data as the second image data.

4. The imaging device of claim 1, wherein the processing unit generates the image data set by arranging combined image data from a plurality of image data including reflected light image data generated from the reflected light luminance data, transmitted light image data generated from the transmitted light luminance data, luminance image data based on the sum of the reflected light luminance data and the transmitted light luminance data, and centroid wavelength image data based on relative values ​​for each wavelength region in the reflected light luminance data and the transmitted light luminance data in a predetermined direction.

5. The imaging device according to any one of claims 1 to 4, wherein the processing unit generates the image data set including identification information for identifying a boundary of the combined image data.

6. An imaging device according to any one of claims 1 to 5, wherein the first image sensor and the second image sensor are the same image sensor.

7. An imaging method comprising: a luminance data generation step of detecting, at a predetermined frame rate, the luminance of reflected light reflected from incident light on a filter surface whose transmittance and reflectance change monotonically in a predetermined wavelength range, to generate reflected light luminance data, and detecting, at the predetermined frame rate, the luminance of transmitted light from the incident light that has passed through the filter surface, to generate transmitted light luminance data; an image data generation step of generating first image data and second image data based on the reflected light luminance data and the transmitted light luminance data output at the same time; and an image data set generation step of generating an image data set combining image data based on the first image data and the second image data.

8. An imaging method according to claim 7, wherein in the image data set generation step, the image data set is generated by combining the first image data generated from the reflected light luminance data and the second image data generated from the transmitted light luminance data.

9. An imaging method as described in claim 7, wherein in the image data set generation step, luminance image data based on the sum of the reflected light luminance data and the transmitted light luminance data is combined as the first image data with centroid wavelength image data based on relative values ​​for each wavelength region in the reflected light luminance data and the transmitted light luminance data as the second image data to generate the image data set.

10. An imaging method as described in claim 7, wherein in the image data set generation step, the image data set is generated by arranging combined image data from a plurality of image data including reflected light image data generated from the reflected light luminance data, transmitted light image data generated from the transmitted light luminance data, luminance image data based on the sum of the reflected light luminance data and the transmitted light luminance data, and centroid wavelength image data based on relative values ​​for each wavelength region in the reflected light luminance data and the transmitted light luminance data in a predetermined direction.

11. An imaging method according to any one of claims 7 to 10, wherein in said image data set generating step, said image data set is generated including identification information for identifying a boundary of the combined image data.

12. A first prism including a separation surface that reflects a portion of incident light and transmits a portion of the incident light, an input surface that inputs a first light reflected at the separation surface, and an output surface that outputs the first light reflected at the input surface; a second prism that is optically coupled to the first prism via the separation surface and includes an output surface that outputs a second light that has transmitted through the separation surface; a filter surface that has a characteristic in which transmittance and reflectance change monotonically in a predetermined wavelength region and is formed on any of the output surface of the first prism, the input surface of the first prism, and the output surface of the second prism; and a first image sensor that detects the luminance of the first light, the first transmitted light that has transmitted through the filter surface formed on the output surface of the first prism, or the first reflected light that has been reflected at the filter surface formed on the input surface of the first prism at a predetermined frame rate and outputs first luminance data. an imaging device comprising: a second image sensor that detects the luminance of the second light or the second transmitted light that has passed through the filter surface formed on the output surface of the second prism at the predetermined frame rate and outputs second luminance data at the same timing as the first luminance data; and a processing unit that generates first image data from the first luminance data, generates second image data from the second luminance data output at the same timing as the first luminance data, and generates an image data set that combines image data based on the first image data and the second image data.

Images