Spectral device with image observation function

Abstract

To deal with the situation where conducting a Raman microscope observation with an optical microscope requires the optical microscope to have a plurality of filter turrets and it is not therefore easy to conduct a Raman microscope observation with an ordinary fluorescent microscope and a special microscope is needed.SOLUTION: There is provided a spectral device with an image observation function, connectable to at least one camera port generally equipped to an optical microscope and being capable of conducting a spectral measurement of Raman scattered light.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] Raman spectroscopy is a method that can obtain chemical properties in a label-free and non-destructive manner. By irradiating a laser beam for local excitation using the objective lens of a microscope, the chemical properties of a sample can be evaluated non-destructively with a spatial resolution corresponding to the optical resolution of the objective lens. Therefore, in the field of drug discovery, by observing the treatment target site over time using Raman spectroscopy from the time of dosing, it is possible to observe when the administered drug reaches the target site and what chemical changes occur.

[0002] Fig. 1 shows a schematic structural diagram of a fluorescence microscope commonly used in the field of biological research. In the fluorescence microscope system 20, the illumination light beam 8 emitted from a lamp light source 1 such as a mercury lamp is guided to the objective lens 3 through the fluorescence filter cube 11 in the filter turret 15 by the condenser lens 2, and illuminates a cell sample 19 disposed together with a cell culture solution 18 in, for example, a culture vessel 16. An excitation filter 12 is disposed in the fluorescence filter cube 11 to select a wavelength suitable for illuminating the fluorescent reagent, and an emission filter 14 is disposed so that the fluorescence wavelength emitted from the fluorescent reagent can be observed. A dichroic mirror 13 having reflection and transmission wavelength characteristics that reflect the illumination light toward the objective lens 3 side and transmit the fluorescence wavelength toward the camera 6 side is disposed. The fluorescence microscope system 20 is equipped with a filter turret 15 on which a plurality of fluorescence filter cubes are mounted so that a filter cube suitable for observation can be easily selected from the plurality of fluorescence filter cubes.

[0003] The fluorescence microscope system 20 is provided with an imaging lens 4 and a camera 6, and can acquire image information in which a cell sample, which is an observation target corresponding to the observation magnification of the objective lens 3, emits fluorescence. The fluorescence microscope system 20 is provided with a camera port 5 so that a camera corresponding to various needs of an observer can be attached. In the fluorescence microscope system 20 shown in FIG. 1, the observation light 9 of the microscope observed through the cover glass 17 becomes an observation light beam 9 of a predetermined wavelength component by the dichroic mirror 13 and the emission filter 14 of the fluorescence filter cube 11, is converged by the imaging lens 4, reflected by the mirror 10, and then forms an image on the imaging element 7 in the camera 6 connected to the camera port 5. And, in order for a camera corresponding to various needs of an observer to be attached to the optical microscope, almost every optical microscope is provided with one or a plurality of camera ports of a standard called a C-mount. The C-mount standard is a standard in which a screw with an inner diameter of 25.4 mm (1 inch) and a pitch of 0.794 mm is formed at the connection part, and the imaging position is arranged at a distance of 17.526 mm from the end face of the screw.

[0004] FIG. 2 shows a configuration example of a Raman microscope system 40 as an example of the construction of an optical system capable of performing a microscopic observation method using the Raman spectroscopy method shown in Patent Document 1. In the Raman spectroscopy method, it is a method of acquiring the substance information from the information on how much the wave number has changed from the wavelength of the illumination light in the wavelength of the light scattered and emitted from the sample. Therefore, for the Raman microscope system, it is effective to use a laser light source having as narrow a wavelength width as possible for the light source irradiating the sample, and a spectroscope is used to acquire the wavelength information of the light scattered and emitted from the sample. Further, the light irradiating the sample is condensed through the objective lens, and in the system shown in FIG. 2, it has a scanning mechanism for two-dimensionally scanning the condensing position. Using a laser light source for the light source, receiving the signal light using a spectroscope, and having a scanning mechanism are different from the fluorescence microscope system 20 shown in FIG. 1.

[0005] Even in a Raman microscope system, it is convenient to acquire image information of a sample by the image observation function of a fluorescence microscope and compare the fluorescence microscope observation image with the information acquired by the Raman microscope system. Therefore, in a commercially available Raman microscope observation apparatus, as shown in FIG. 3, a lamp light source 1 and a camera 6 that are not used when acquiring Raman spectroscopic information are generally arranged. In the configuration shown in FIG. 3, the filter cube is not arranged at the position of the filter turret 27, and the filter cube 11 is arranged at the position of the filter turret 15.

[0006] A method for acquiring Raman spectroscopic information of a cell sample 19 disposed together with a cell culture solution 18 in a culture vessel 16 using the Raman microscope system 40 in FIG. 2 will be described. For example, the laser light emitted from the laser light source 21 having a wavelength of 532 nm is collimated by the condenser lens 22, reflected by the mirror 36, and then reflected by the dichroic mirror 23 that reflects the wavelength of the laser light source and transmits a wavelength longer than the wavelength of the laser light source 21, and then irradiated to the two-dimensional movable mirror device 24. The laser light reflected by the mirror 24a in the two-dimensional movable mirror device 24 enters the scan lens 25 composed of an f-theta lens and forms a condensing spot on the imaging surface 37. By arranging the mirror 24a at the back focus position of the scan lens 25, the position of the condensing spot formed on the imaging surface 37 can be controlled according to the angle of the movable mirror 24a. The laser beam 34 that forms a condensing spot on the condensing surface 37 is condensed onto the sample to be observed through the cover glass 17 by the imaging optical system formed by the imaging lens 26 and the objective lens 3. Here, in the Raman microscope system 40, in order to have a fluorescence microscope observation function, a mirror 29 that can be retracted from the optical system is arranged in the filter cube 28 arranged in the second filter turret 27 between the imaging lens 26 and the objective lens 3.

[0007] The scattered light beam 35 emitted from the sample is collected by the objective lens 3, focused on the imaging plane 37 by the mirror 29 and the imaging lens 26, then made into a substantially parallel light beam by the scan lens 25, and then reflected by the movable mirror 24a and reaches the dichroic mirror 23. Since the dichroic mirror 23 has the wavelength characteristic of transmitting wavelengths longer than that of the laser light source 21 as described above, the scattered light of the wavelength component longer than that of the laser light source 21 passes through the dichroic mirror 23, absorbs the light of the laser light source 21, passes through the laser light absorption filter 30 that transmits wavelengths longer than that of the laser light source 21, then is incident on the multimode optical fiber 32 by the condenser lens 31, then is incident on the spectroscope 33, the wavelength component information of the scattered light is measured, Raman spectroscopic information is obtained, and the chemical characteristics of the sample can be obtained.

[0008] Next, a method for obtaining a fluorescence observation image of the cell sample 19 disposed together with the cell culture solution 18 in the culture vessel 16 using the Raman microscope system 40 will be described with reference to FIG. 3. For the illumination of the fluorescence microscope observation, the illumination light beam 8 emitted from the lamp light source 1 needs to pass through the excitation filter 12 in the fluorescence filter cube 11 provided in the first filter turret 15, be reflected by the dichroic mirror 13, and be incident on the objective lens 3. Since the fluorescence light beam 9 of the cell sample 19 is incident on the camera 6 through the imaging lens 4, the filter cube 28 provided in the second filter turret 27 needs to be retracted during the fluorescence microscope observation.

[0009] The fluorescence filter cube 11 provided within the first filter turret 15 and the filter cube 28 provided within the second filter turret 27 both need to be retractable. Also, since the first filter turret 15 is connected to the optical path of the lamp light source 1 and the second filter turret 27 is connected to the optical path of the laser light source 21, both the second filter turret 27 and the first filter turret 15 need to be arranged on the optical axis of the objective lens 3, but it is difficult to make them share the same function. Since the number of filter turrets provided in a fluorescence microscope is usually one, it is not easy to add a Raman microscope function to a normal microscope, and it is necessary to perform a modification to make the filter turret a two-stage configuration.

[0010] Furthermore, in the Raman microscope system 40 shown in FIGS. 2 and 3, in order to obtain the positional information relationship between the image acquired by the camera 6 and the laser focusing spot irradiated by the driving of the movable mirror 24, after constructing the Raman microscope system 40, it is necessary to perform Raman spectroscopic information of the sample, image shooting by the camera 6, compare the data, and perform position calibration. Therefore, when the camera replacement work is performed, it is necessary to perform the position calibration work of the Raman spectroscopic information of the sample and the image shooting by the camera 6 each time.

Prior Art Documents

Patent Documents

[0011]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0012] The present invention has been made in consideration of the above points. In order to perform Raman microscope observation using a fluorescence microscope, since the fluorescence microscope needs to have a plurality of filter turrets, the present invention provides a solution to the problem that it is not easy to perform Raman microscope observation with a normal fluorescence microscope.

Means for Solving the Problem

[0013] In order to solve such a problem, in the spectroscopic apparatus with an image observation function of the present invention, a spectroscopic apparatus with an image observation function capable of spectroscopic measurement of Raman scattered light is configured to be connectable to one or a plurality of camera ports generally provided in an optical microscope.

Effect of the Invention

[0014] In the spectroscopic apparatus with an image observation function of the present invention, it is a system that can be easily attached without modifying a general optical microscope, and it is possible to acquire Raman spectroscopic information, so that Raman spectroscopic information can be acquired at low cost. Further, when a plurality of microscopes such as an upright fluorescence microscope and an inverted fluorescence microscope are owned, it is possible to acquire Raman spectroscopic information with a plurality of microscopes by a simple removal process and attachment process that are almost the same as replacing a camera.

[0015] Furthermore, in a Raman microscope system in which the spectroscopic apparatus with an image observation function of the present invention and a camera are connected to different camera ports using a microscope equipped with a plurality of camera ports, although an alignment process between an image taken by the camera and two-dimensional Raman spectroscopic information is required, by aligning the image sensor provided in the spectroscopic apparatus with an image observation function of the present invention and the camera, it is possible to align an image taken by the camera and two-dimensional Raman spectroscopic information, so that alignment can be performed without the need to acquire Raman spectroscopic information.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Embodiments for Carrying Out the Invention

[0017] Fig. 4 shows a schematic configuration diagram of a spectroscopic apparatus 100 with an image observation function as a first embodiment of the present invention. In the spectroscopic apparatus with an image observation function of the present invention, it has a camera port connection port 5a corresponding to the C-mount standard. When connected to the C-mount camera port of a microscope, an image sensor 7 is arranged at the position where the microscope observation light beam 9 of the sample forms an image. Between the camera port connection port 5a and the image sensor 7, a dichroic mirror 46 that selectively reflects light with a laser wavelength of the laser light source 41 and a wavelength longer than the laser wavelength of the laser light source 41 is arranged. Here, the dichroic mirror 46 is a plate-shaped one with a thickness of about 1 mm. When a plate material is arranged obliquely in the converging optical system, astigmatism is generated. However, since the focal length of the imaging lens of the optical microscope is approximately 200 mm or about 180 mm, a dichroic mirror 46 made of a glass material with a refractive index of about 1.5 and a thickness of 2 mm or less, preferably about 1 mm, is arranged in front of the image sensor 7 at an angle of 45 degrees, and an image can be taken without the influence of astigmatism in the captured image by the image sensor 7. Furthermore, since astigmatism only affects the optical path that passes through the glass material, by arranging the reflective coating surface of the dichroic mirror 46 on the side of the scan lens 45, the laser light beam 54 is not affected by astigmatism. Therefore, the laser light formed on the sample when acquiring spectroscopic information is not affected by astigmatism. By using a cube-shaped dichroic mirror for the dichroic mirror 46, it is also possible to reduce the astigmatism of the image captured by the image sensor 7. However, the plate-shaped dichroic mirror is less expensive than the cube-shaped dichroic mirror.

[0018] The narrow-line laser light with a wavelength of, for example, 532 nm emitted from the laser light source 41 is collimated by the condenser lens 42, then reflected by the mirror 56, and further reflected by the dichroic mirror 43 that reflects the wavelength of the laser light source and transmits a wavelength longer than that of the laser light source 41, and is irradiated onto the mirror 64. The laser light reflected by the mirror 64 enters the scan lens 45 and forms a condensing spot on the imaging surface 57. The imaging surface 57 is an imaging surface formed by the light reflected by the dichroic mirror 46, by inserting the dichroic mirror 46 between the camera mount connection port 5a and the imaging element 7. Also, the scattered light from the sample having a wavelength component longer than that of the laser light source 41 forms a condensing spot on the imaging surface 57, then passes through the scan lens 45 and the mirror 64, then passes through the dichroic mirror 43, then passes through the laser light absorption filter 50 that absorbs the light of the laser light source 41 and transmits a wavelength longer than that of the laser light source 41, then enters the multimode optical fiber 52 by the condenser lens 51, then enters the spectroscope 53, and the wavelength component information of the scattered light is measured, Raman spectroscopic information is obtained, and the chemical characteristics of the sample can be obtained.

[0019] Here, in the spectroscopic apparatus 100 with an image observation function, the spectroscope 53 uses the flexibility of the optical fiber 52 and is optically connected but has a movable configuration from the portion 101 integrated with the camera port connection port 5a in the spectroscopic apparatus 100 with an image observation function. Specifically, the incident-side connector portion 52a of the multimode optical fiber 52 is arranged in the portion 101 integrated with the camera port connection port 5a, the incident-side connector portion 52c of the multimode optical fiber 52 is connected to the spectroscope 53 and arranged at a position having a distance of, for example, 1 m from the portion 101 integrated with the camera port connection port 5a, and is connected by the fiber portion 52b of the 2-m multimode optical fiber 52. With this configuration, it is possible to reduce the weight and volume of the portion 101 integrated with the camera port connection port 5a, and it is also possible to prevent mechanical interference when connecting the spectroscopic apparatus 100 with an image observation function to an optical microscope.

[0020] FIG. 5 shows a Raman microscope system 102 realized by connecting a spectroscopic apparatus 100 with an image observation function shown in FIG. 4 to a camera port 5 of the fluorescence microscope 20 shown in FIG. 1. Since the imaging element 7 is arranged on the imaging plane when connected to the camera mount 5 of the C-mount standard, if appropriate illumination such as transmitted illumination or illumination for autofluorescence observation with ultraviolet excitation is applied to the observation object in the observation field of the objective lens 3, an image of the observation object can be taken by the imaging element 7.

[0021] From the description of the spectroscopic apparatus 100 with an image observation function using FIG. 4 above, by arranging a dichroic mirror 46 between the camera port 5 and the imaging element 7 and placing the condensing spot of the laser beam 54 emitted from the laser light source 41 on the imaging plane 57 generated thereby, the laser light can be condensed and irradiated at a position 60 corresponding to the condensing spot position 58 of the laser light source 41 on the imaging plane 57 of the observation object in the observation field of the objective lens 3. The scattered light emitted from the sample by condensing and irradiating the laser light also forms a spot at the position 58 of the imaging plane 57, then passes through the dichroic mirror 43 and the laser light absorption filter 50, is incident on the multimode optical fiber 52 by the condenser lens 51, is incident on the spectroscope 53, the wavelength component information of the scattered light is measured, Raman spectroscopic information is acquired, and the chemical characteristics of the sample can be acquired. Therefore, the spectroscopic apparatus 100 with an image observation function shown in FIG. 4 can acquire the observation image using the imaging element 7 and the spectroscopic characteristics of the scattered light emitted by condensing and irradiating the laser light emitted from the laser light source 41 by connecting to the camera port 5 of the optical microscope, so that the chemical characteristics of the sample can be acquired.

[0022] Here, as a feature of the spectroscopic apparatus 100 with an image observation function, the portion 101 integrated with the camera port connection port 5a shown by the dashed line in FIGS. 4 and 5 does not change its relative position regardless of whether the spectroscopic apparatus 100 with an image observation function is connected to the camera port 5 or not. Therefore, the condensing spot position 58 of the laser light source 41 on the imaging surface 57 and the position 59 where the laser light condensing irradiation position 60 on the sample is observed by the imaging device 7 are constant. Therefore, even if the sample is moved or the spectroscopic apparatus 100 with an image observation function is removed and connected to the camera port of another microscope, the position 59 of the laser spot irradiated on the sample in the sample image observed by the imaging device 7 remains unchanged.

[0023] FIG. 6 shows an example of an image in which a marker 72 is superimposed on the image of the sample taken by the imaging device 7 of the spectroscopic apparatus 100 with an image observation function at a position corresponding to the position 59 of the laser spot. In this image, the outer shape of the cell 19 is shown as a line 71. By indicating the laser irradiation position with a marker in the taken image and superimposing and displaying the marker on the image, the user can easily understand which position of the sample the spectroscopic characteristic information has been obtained without performing an alignment calibration operation after attaching the spectroscopic apparatus 100 with an image observation function to the microscope. Also, by superimposing and saving the laser irradiation position on the taken image using a marker together with the spectroscopic data obtained by the spectroscope, the user can easily save which position the spectroscopic data has been measured.

[0024] Fig. 7 shows a schematic configuration diagram of a spectroscopic apparatus 103 with an image observation function as a second embodiment of the present invention. In the spectroscopic apparatus 103 with an image observation function as a second embodiment of the present invention, it has a camera port connection port 5a corresponding to the C-mount standard, and when connected to the C-mount camera port of a microscope, an imaging element 7 is arranged at the position where the microscopic observation light beam 9 of the sample forms an image. A dichroic mirror 46 that selectively reflects the laser wavelength of the laser light source 41 and light with a wavelength longer than the laser wavelength of the laser light source 41 is arranged between the camera port connection port 5a and the imaging element 7. Here, the dichroic mirror 46 has a plate shape of about 1 mm. Although astigmatism is generated when a plate material is obliquely arranged in the converging optical system, since the focal length of the imaging lens of the optical microscope is about 200 mm or about 180 mm, a dichroic mirror 46 with a plate shape of about 1 mm made of a glass material with a refractive index of about 1.5 is arranged in front of the imaging element 7 at an angle of 45 degrees, and an image can be taken without the influence of astigmatism in the captured image by the imaging element 7. Furthermore, since astigmatism only affects the optical path passing through the glass material, by arranging the reflective coating surface of the dichroic mirror 46 on the side of the scan lens 45, the laser light beam 54 is not affected by astigmatism, so the laser light formed on the sample when acquiring spectroscopic information is not affected by astigmatism.

[0025] For example, the narrow-line laser light with a wavelength of 532 nm emitted from the laser light source 41 is made parallel by the condenser lens 42, then reflected by the mirror 56, and further reflected by a dichroic mirror 43 that reflects the wavelength of the laser light source and transmits a wavelength longer than the wavelength of the laser light source 41, and is irradiated onto the movable mirror 44a of the movable mirror system 44. The laser light reflected by the mirror 44a enters the scan lens 45 and forms a condensing spot on the imaging surface 57. The imaging surface 57 is the imaging surface formed by the light reflected by the dichroic mirror 46 by inserting the dichroic mirror 46 between the camera mount connection port 5a and the imaging element 7. By arranging the movable mirror 44a at the back focus position of the scan lens 45, the position of the condensing spot formed on the imaging surface 57 can be controlled according to the angle of the movable mirror 44a.

[0026] In addition, the scattered light from the sample with a wavelength component longer than the wavelength of the laser light source 41 forms a condensing spot on the imaging surface 57, then passes through the scan lens 45 and the mirror 64, then passes through the dichroic mirror 43, and then passes through a laser light absorption filter 50 that absorbs the light of the laser light source 41 and transmits a wavelength longer than the wavelength of the laser light source 41. After being incident on the multimode optical fiber 52 by the condenser lens 51, it is incident on the spectroscope 53, the wavelength component information of the scattered light is measured, the Raman spectroscopic information is acquired, and the chemical characteristics of the sample can be acquired.

[0027] Here, in the spectroscope 53 of the spectroscopic apparatus 103 with an image observation function, using the flexibility of the optical fiber 52, in the spectroscopic apparatus 103 with an image observation function, from the portion 104 integrated with the camera port connection port 5a, although optically connected, it has a movable configuration. Specifically, the incident-side connector portion 52a of the multimode optical fiber 52 is arranged in the portion 104 integrated with the camera port connection port 5a, the incident-side connector portion 52c of the multimode optical fiber 52 is connected to the spectroscope 53 and arranged at a position having a distance of, for example, 1 m from the portion 104 integrated with the camera port connection port 5a, and is connected by the fiber portion 52b of the 2-m multimode optical fiber 52 and the like. With this configuration, it is possible to reduce the weight and volume of the portion 104 integrated with the camera port connection port 5a, and it is also possible to prevent mechanical interference and the like when connecting the spectroscopic apparatus 103 with an image observation function to an optical microscope.

[0028] FIG. 8 shows a Raman microscope system 105 realized by connecting the spectroscopic apparatus 103 with an image observation function shown in FIG. 7 to the camera port 5 of the fluorescence microscope 20 shown in FIG. 1. Since the imaging element 7 is arranged on the imaging plane when connected to the camera mount 5 of the C-mount standard, if appropriate illumination such as transmitted illumination or illumination for autofluorescence observation with ultraviolet excitation is applied to the observation object within the observation field of view of the objective lens 3, it becomes possible to capture an image of the observation object with the imaging element 7.

[0029] From the description of the spectroscopic apparatus 103 with an image observation function using the above-mentioned figure, by arranging the dichroic mirror 46 between the camera port 5 and the imaging element 7 and placing the condensing spot of the laser beam 54 emitted from the laser light source 41 on the imaging plane 57 generated thereby, the laser light can be focused and irradiated at a position 60 corresponding to the position of the condensing spot of the laser light source 41 on the imaging plane 57 of the observation object within the observation field of view of the objective lens 3. The scattered light emitted from the sample by focusing and irradiating the laser light also forms a spot on the imaging plane 57, then passes through the dichroic mirror 43 and the laser light absorption filter 50, is incident on the multimode optical fiber 52 by the condenser lens 51, and then is incident on the spectroscope 53, where the wavelength component information of the scattered light is measured, Raman spectroscopic information is acquired, and the chemical characteristics of the sample can be obtained.

[0030] Therefore, the spectroscopic apparatus 103 with an image observation function shown in Fig. 7 can, by connecting to the camera port 5 of the optical microscope, acquire an observation image using the imaging element 7 and obtain the spectroscopic characteristics of the scattered light emitted by focusing and irradiating the laser light emitted from the laser light source 41, so that the chemical characteristics of the sample can be obtained. Furthermore, by adjusting the angle of the movable mirror 44a, a laser irradiation spot can be formed at an arbitrary position within the observation field of view of the objective lens 3, and the chemical characteristics of the laser irradiation position of the sample can be obtained.

[0031] Fig. 9 shows an example of specifying points for acquiring a plurality of chemical properties using an image captured by the imaging device 7. In the image captured by the imaging device 7 shown in Fig. 9, the cell position is indicated by line 71, and the figure shows a case where the user has specified, for example, two laser irradiation positions 70 in the image. The system of the present invention can display and save the image shown in Fig. 9. As shown in Fig. 9, by indicating the laser irradiation position with a marker on the captured image and superimposing and displaying the marker on the image, the user can easily understand which position of the sample the spectroscopic characteristic information has been acquired from, without having to perform an alignment calibration operation after attaching the spectroscopic apparatus 103 with an image observation function to the microscope. Also, by superimposing and saving the image with the laser irradiation position marked on the captured image using a marker together with the spectroscopic data acquired by the spectroscope, the user can easily save which position the spectroscopic data has been measured from.

[0032] Fig. 10 shows an example of specifying a region for acquiring chemical properties using an image captured by the imaging device 7. In the image captured by the imaging device 7 shown in Fig. 10, the cell position is indicated by line 71, and the figure shows a case where the user has specified a region 73 formed of a polygon that surrounds, for example, the left cell in the image. The system of the present invention can display and save the image shown in Fig. 10. As shown in Fig. 10, by indicating the laser irradiation position with a marker 73 on the captured image and superimposing and displaying the marker on the image, the user can easily understand which position of the sample the spectroscopic characteristic information has been acquired from, without having to perform an alignment calibration operation after attaching the spectroscopic apparatus 103 with an image observation function to the microscope. Also, by superimposing and saving the image with the laser irradiation position marked on the captured image using a marker together with the spectroscopic data acquired by the spectroscope, the user can easily save which position the spectroscopic data has been measured from.

[0033] FIG. 11 is a diagram showing a method of creating position information for acquiring chemical characteristics of a laser irradiation position of a sample after a region 73 composed of a specified polygon shown in FIG. 10 is specified. Since it is difficult to acquire the spectroscopic characteristics of scattered light generated by laser irradiation in units of milliseconds with a spectroscope, it is desirable that the density of data acquisition points be changeable according to user needs. Therefore, in the spectroscopic apparatus 103 with an image observation function of the present invention, in the image captured by the imaging device 7, the user can specify the interval between the measurement positions in terms of the number of pixels of the image. In FIG. 11, the size of the pixel is shown as a square, and the pixels 76 filled with color representing the positions specified as the positions for acquiring the chemical characteristics of the laser irradiation position of the sample by the specification of the region 73, and the pixels 77 not filled with color representing the pixels not specified are shown. That is, an example is shown in which the pixel interval specified by the user is set to be every 4 pixels as the intervals indicated by arrows 74 and 75. After the positions of the pixels 76 for performing laser irradiation within the specified region 73 are determined, the position of the movable mirror 44a is adjusted so that the laser irradiation spots are sequentially irradiated to the positions of the pixels 76.

[0034] Using FIGS. 12, 13, and 14, an explanation will be given of a calibration method between the input voltage value used for controlling the movable mirror 44a and the pixel information of the image and a method of calculating the input voltage value used for control in order to move the laser irradiation spot to a predetermined position of the image captured by the imaging device 7.

[0035] FIG. 12 is a diagram showing a luminescent material 80 that emits light of a wavelength that can be captured by an imaging device 7 by irradiating a sample position with a laser beam having a wavelength emitted from a laser light source 41 in a Raman microscope system 105 realized by connecting a spectroscopic device 103 with an image observation function shown in FIG. 7 to a camera port 5 of the fluorescence microscope 20 shown in FIG. 1. The luminescent material 80 is disposed via a cover glass 79 of an objective lens. By using, for example, a viewing card VRC1 sold by Solarbo Japan Co., Ltd., the portion irradiated with light having a wavelength of 532 nm emits light in the wavelength range of 450 nm to 750 nm, so that the irradiation position of the laser beam can be captured as a bright point by the imaging device 7.

[0036] When, for example, a mirror device (model number: A8L2.2 - 5000AL - TINY48.4 - A / W / TP) capable of two - dimensional driving in the X and Y directions of mirrorcle technologies is used as the movable mirror 44a and controlled by an analog control board (model number: DR - 11 - 055 - 00), both the X - axis and the Y - axis have the characteristic that the angle linearly changes from - 4 degrees to + 4 degrees with respect to an input voltage signal from - 10V to + 10V. FIG. 13 shows the position of the light - emitting point photographed by the imaging device 7 when the voltage value in the X direction of the movable mirror is 0V and the voltage value in the Y direction of the movable mirror is 0V as 81, the position of the light - emitting point photographed by the imaging device 7 when the voltage value in the X direction of the movable mirror is 1V and the voltage value in the Y direction of the movable mirror is 0V as 82, and the position of the light - emitting point photographed by the imaging device 7 when the voltage value in the X direction of the movable mirror is 0V and the voltage value in the Y direction of the movable mirror is 1V as 83. Here, the number of pixels of the imaging device 7 is 1920 pixels and 1080 in the horizontal direction (X direction) and the vertical direction (Y direction), respectively. And the coordinates of point 81, point 82, and point 83 are point 81: (960, 540), point 82: (1160, 540), and point 83: (960, 390).

[0037] The angle of the movable mirror 44a changes independently and linearly with respect to voltage inputs in the X and Y directions. That is, when a voltage of 1 V is applied in the X direction, it moves 200 pixels, and when a voltage of 1 V is applied in the Y direction, it moves -150 pixels. And as shown in FIG. 14, for any point 84 in the image captured by the imaging device 7, the voltage value to be applied can be obtained by expressing the vector 88 from point 81 to point 84 using the vector 86 from point 81 to point 82 and the vector 87 from point 81 to point 83. Specifically, assuming that the pixel coordinates of point 84 are (X1, Y1), the voltage value VX applied to the movable mirror 44a in the X direction is (X1 - 960) / 200 (V), and the voltage value VY applied in the Y direction is (Y1 - 540) / (-150) (V).

[0038] Here, as a feature of the spectroscopic apparatus 103 with an image observation function, the portion 104 integrated with the camera port connection port 5a shown by the broken line in FIGS. 7 and 8 does not change its relative position whether the spectroscopic apparatus 103 with an image observation function is connected to the camera port 5 or not. Therefore, the positional relationship of the condensing spot of the laser light source 41 on the imaging surface 57 when the angle of the movable mirror 44a is changed, and the positional relationship where the laser light condensing irradiation position on the sample is observed by the imaging device 7 are constant. Therefore, even if the sample is moved, or if the spectroscopic apparatus 103 with an image observation function is removed and connected to the camera port of another microscope, the relationship between the position of the laser spot irradiated on the sample in the sample image observed by the imaging device 7 and the voltage value input to the movable mirror 44a remains unchanged. Thus, it becomes unnecessary to perform the calibration work for each attachment to the microscope.

[0039] FIG. 15 shows a second Raman microscope system 106 realized by connecting the spectroscopic apparatus 103 with an image observation function shown in FIG. 7 and the camera 96 to a fluorescence microscope having a plurality of camera ports. The spectroscopic apparatus 103 with an image observation function is connected to the first camera port 5 provided in the fluorescence microscope 20 shown in FIG. 1, and the camera 96 is connected to the second camera port 95 added to the fluorescence microscope 20. Here, both of the two camera ports 5 and 95 are camera ports conforming to the C-mount standard.

[0040] In the Raman microscope system 106 shown in FIG. 15, it is possible to specify a position or region for acquiring the spectroscopic characteristics of scattered light that emits light by condensing and irradiating the laser light emitted from the laser light source 41, as well as for acquiring an observation image by the camera 96. In front of the image sensor 7 in the spectroscopic apparatus 103 with an image observation function, a dichroic mirror 46 that reflects the wavelength of the laser light emitted from the laser light source 41 and a wavelength longer than the wavelength of the laser light is arranged. Therefore, there are restrictions on the wavelength range in the image captured by the image sensor 7. However, between the image sensor 97 in the camera 96 connected to the second camera port 95 of the Raman microscope system 106 and the imaging lens 94 corresponding to the second camera port 95, there is no member that restricts the transmission wavelength. Therefore, there is no wavelength limitation regarding the captured image for the camera 96. Therefore, it is possible to capture an image even when using transmitted illumination with a wavelength longer than the wavelength of the laser light source 41.

[0041] Using FIGS. 15, 16, and 17, the method for acquiring position calibration data necessary when specifying a position or region for acquiring the spectroscopic characteristics of scattered light that emits light by condensing and irradiating the laser light emitted from the laser light source 41 using the observation image captured by the camera 96 will be described.

[0042] As described above, in the spectroscopic apparatus 103 with an image observation function of the present invention, the positional relationship of the condensing spot of the laser light source 41 on the imaging plane 57 when the angle of the movable mirror 44a is changed, and the positional relationship of the position where the laser light is condensed and irradiated on the sample being observed by the image sensor 7 are constant. Therefore, by calibrating the positional relationship between the camera 96 and the camera 6, it is possible to acquire the positional relationship between the camera 96 and the angle of the movable mirror 44a. Specifically, even without performing image capturing by the camera 6, it becomes possible to obtain the angle information of the movable mirror 44a based on the position information in the captured image of the camera 6 for the position in the captured image of the camera 96. Therefore, when obtaining the positional relationship between the camera 96 and the angle of the movable mirror 44a, it is not necessary to perform operations such as emitting light from the laser light source 41 and changing the angle of the movable mirror 44a.

[0043] The calibration operation between the position information in the captured image of the camera 96 and the position information in the captured image of the camera 6 can be achieved by, as shown in FIG. 15, arranging a captured sample 91 having a characteristic pattern in the field of view of the objective lens 3, adjusting the focal position of the objective lens 3, and performing operations of capturing by the camera 6 and capturing by the camera 96.

[0044] An example of a captured image by the camera 6 is shown in Fig. 16, and an example of a captured image by the camera 96 is shown in Fig. 17. In Fig. 16 and Fig. 17, the characteristic patterns of the sample 91 are images according to the pixel size and the number of pixels of each camera. By comparing these images, it can be understood that the feature points 61, 62, 63 in Fig. 16 correspond to the feature points 64, 65, 66 in Fig. 17. Therefore, as described with reference to Figs. 13 and 14, for two non-parallel vectors, that is, for any point in the image captured by the camera 96, the coefficients are obtained using the vector from point 64 to point 65 and the vector from point 64 to point 66 in Fig. 17, and in the image captured by the camera 6, the obtained coefficients are given to the vector from point 61 to point 62 and the vector from point 61 to point 63 in Fig. 16, respectively, so that it is possible to determine which pixel in the image captured by the camera 6 corresponds. Specifically, if the point 68 in Fig. 17 captured by the camera 96 can be expressed as the sum of the vector obtained by multiplying the vector from point 64 to point 65 by 0.6 and the vector obtained by multiplying the vector from point 64 to point 66 by 0.8, then the point 68 in Fig. 16 captured by the camera 96 corresponds to the pixel expressed as the sum of the vector obtained by multiplying the vector from point 61 to point 62 by 0.6 and the vector obtained by multiplying the vector from point 61 to point 63 by 0.8 in Fig. 16 captured by the camera 6.

[0045] As described above, in the spectroscopic apparatus 103 with an image observation function of the present invention, when acquiring the positional relationship between the camera 96 and the angle of the movable mirror 44a, there is no need to perform operations such as the emission of the laser light source 41 and the angle change of the movable mirror 44a, and it has the feature that it can be easily performed.

[0046] In the above description of the present invention, an example where the camera ports 5 and 95 are the most popular C-mounts is shown. However, in an optical microscope, since there may be standards for each optical microscope manufacturer, the camera mount in the present invention is not limited to the C-mount. In the description of the present invention above, the form of connecting the spectroscopic apparatus with an image observation function of the present invention to the camera port of an optical microscope has been described. However, the optical system for taking an image of a sample is not limited to a magnifying optical system, and it may be a stereomicroscope having a camera port such as a C-mount. Further, it may be a single lens having a connection portion such as a C-mount.

Industrial Applicability

[0047] The spectroscopic apparatus with an image observation function of the present invention can be used for analysis of a sample by Raman spectroscopy, which is a method capable of obtaining chemical characteristics in a label-free and non-destructive manner. And, although the field of use of the spectroscopic apparatus with an image observation function of the present invention is not limited to the biological field, in the field of drug discovery and the like, by observing the target site over time from the time of drug administration, it can be used to observe how the drug from administration reaches the target site and what chemical changes occur.

Explanation of Signs

[0048] 1... lamp light source, 2, 22, 42... condenser lens, 3, 31, 51... objective lens, 4, 26, 94... imaging lens, 5, 95... camera port, 6, 96... camera, 7, 97... imaging element, 8... illumination light beam, 9... observation light beam, 10, 29, 36, 56, 59, 90... mirror, 11, 28... fluorescence filter cube, 12... excitation filter, 13, 23, 43... dichroic mirror, 14... emission filter, 15, 27... filter turret, 16... observation container, 17, 79... cover glass, 18... culture solution, 19... cell sample, 20... fluorescence microscope system, 21, 41... laser light source, 24, 44... movable mirror, 25, 45... scan lens, 30, 50... laser light absorption filter, 32, 52... optical fiber, 33, 53... spectroscope, 34, 54... laser light beam, 35, 55... Raman spectrobeam, 37, 57... imaging plane, 40, 102, 105, 106... Raman microscope system, 58, 60... condenser spot position, 59... position of the imaging element corresponding to the condenser spot position, 61, 62, 63, 64, 65, 66... feature points of the photographed sample, 67, 68... designated point position, 70... laser irradiation designated point, 71... cell sample image, 72... laser irradiation position marker, 73... laser irradiation designated area, 74, 75... designated grid interval, 76... selected laser irradiation position, 77... unselected laser irradiation position, 80... luminescent material, 81, 82, 83, 84... laser irradiation points, 86, 87, 88... vectors, 91... photographed sample, 100, 103... microscope-mounted Raman spectrometer, 101, 104... Raman spectrometer microscope mounting part

Claims

1. It is detachable from the image observation optical system, a laser light source that excites scattered light, a spectroscope that acquires wavelength information of scattered light from a sample, A spectroscopic device with an image observation function having an image sensor that acquires image information, The image sensor is arranged at a first imaging position of an image of a sample observed by the image observation optical system, A first spectroscopic mirror that reflects light having a wavelength of the laser light source and a wavelength longer than the wavelength of the laser light source is arranged between the detachable part to the image observation optical system and the image sensor, A first optical lens is arranged at a second imaging position formed by arranging the first spectroscopic mirror so that the laser light that excites laser scattered light is condensed and the laser light irradiates the sample through the image observation optical system, By arranging a second spectroscopic mirror that reflects the laser light and separates the scattered light so that the scattered light from the sample is condensed at the second imaging position through the image observation optical system and the first spectroscopic mirror, and then made into substantially parallel light by the first optical lens and incident on the spectroscope, A spectroscopic device with an image observation function, characterized in that it acquires image information and spectroscopic information of a sample to be observed

2. In the spectroscopic device with the above image observation function, The spectroscopic device with an image observation function according to claim 1, characterized in that information obtained by adding the laser irradiation position where the spectroscopic information of the sample is acquired to the image information of the sample obtained using the image observation optical system is output.

3. In the spectroscopic device with the above image observation function, The spectroscopic device with an image observation function according to claim 1 or 2, characterized in that the position for acquiring spectroscopic information is specified using the image information of the sample obtained using the image observation optical system.

4. In the spectroscopic device with the above image observation function, The spectroscopic device with an image observation function according to any one of claims 1 to 3, characterized in that the image information of the sample obtained using the image observation optical system is image information photographed by an image sensor connected to an image observation optical system different from the image sensor arranged in the spectroscopic device with the image observation function.

5. In the spectroscopic device with the above image observation function, The spectroscopic device with an image observation function according to any one of claims 1 to 4, characterized in that the position of the laser light irradiated onto the sample through the image observation optical system is adjusted by arranging a mirror device with an adjustable angle between the first optical lens and the second spectroscopic mirror.

6. In the spectroscopic device with the above image observation function, By designating a region for acquiring spectroscopic information using the image information of a sample acquired using an image observation optical system, spectroscopic information at a plurality of positions within the corresponding region is acquired. The spectroscopic apparatus with an image observation function according to any one of claims 1 to 5, characterized in that

7. In the spectroscopic apparatus with an image observation function, The first spectroscopic mirror is a plate-shaped member having a thickness of about 2 mm or less, and a coating surface that reflects light having a wavelength longer than the wavelength of the laser light source is disposed on the laser light source side. The spectroscopic apparatus with an image observation function according to any one of claims 1 to 6, characterized in that

8. In the spectroscopic apparatus with an image observation function, The first spectroscopic mirror is a plate-shaped member having a thickness of about 1 mm, and a coating surface that reflects light having a wavelength longer than the wavelength of the laser light source is disposed on the laser light source side. The spectroscopic apparatus with an image observation function according to any one of claims 1 to 6, characterized in that

9. In the spectroscopic apparatus with an image observation function, The image observation optical system to which the spectroscopic apparatus with an image observation function is connected is an optical microscope. The spectroscopic apparatus with an image observation function according to any one of claims 1 to 8, characterized in that

10. In the spectroscopic apparatus with an image observation function, The detachable part of the image observation optical system to the image observation optical system of the spectroscopic apparatus with an image observation function is a C mount. The spectroscopic apparatus with an image observation function according to any one of claims 1 to 9, characterized in that

Images