Spectroscopic measurement method and spectroscopic camera

Abstract

To provide a spectroscopic measurement method and a spectroscopic camera capable of obtaining an accurate spectroscopic spectrum.SOLUTION: A method includes: a step of capturing an image of a subject for each wavelength to be measured; a step of extracting at least three feature points corresponding to a circumference of the subject for each of all captured images captured in the step of capturing; and a step of compensating images by using the feature points for each of the captured images so as to match coordinates of each of the feature points among all the captured images.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a spectroscopic measurement method and a spectroscopic camera. [Background technology]

[0002] Patent Document 1 discloses the configuration of a spectroscopic camera equipped with a wavelength-tunable filter. Because the spectroscopic camera acquires many images while switching wavelengths, it may take several seconds, for example, to capture all the images. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2019-45599 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with mobile spectroscopic cameras, it is difficult for users to hold the spectroscopic camera without moving it for several seconds. Even with fixed spectroscopic cameras, there is a problem that the camera may move due to environmental vibrations, which reduces the analytical accuracy of captured images. Therefore, a practical spectroscopic measurement method for spectroscopic cameras is needed. [Means for solving the problem]

[0005] The spectroscopic measurement method includes a step of capturing an image of a subject for each wavelength to be measured, a step of extracting at least three feature points corresponding to the periphery of the subject for each of all captured images captured in the capturing step, and a step of performing image correction using the feature points of each of the captured images to match the coordinates of each feature point among all of the captured images.

[0006] The spectroscopic camera includes a spectral filter, an optical sensor, an incident optical system, a control unit that controls the transmission wavelength of the spectral filter, a memory unit that stores captured images captured by the optical sensor for each of the transmitted wavelengths, a calculation processing unit that performs calculation processing based on the captured images, a feature point extraction unit that extracts at least three feature points corresponding to the periphery of a subject for each of the captured images, and a position alignment unit that uses the feature points of each of the captured images to match the coordinates of each feature point between all of the captured images. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of a spectroscopic camera. [Figure 2] 3 is a flowchart showing a spectroscopic measurement method according to the first embodiment. [Figure 3] FIG. 4 is a diagram illustrating a part of image correction. [Figure 4] FIG. 4 is a diagram illustrating a part of image correction. [Figure 5] 10 is a flowchart showing a spectroscopic measurement method according to a second embodiment. [Figure 6] FIG. 4 is a diagram illustrating a part of image correction. [Figure 7] FIG. 4 is a diagram illustrating a part of image correction. [Figure 8] FIG. 4 is a diagram illustrating a part of image correction. DETAILED DESCRIPTION OF THE INVENTION

[0008] The configuration of the spectroscopic camera 100 will be described with reference to FIG.

[0009] 1, the spectroscopic camera 100 can obtain a highly accurate spectroscopic spectrum at a predetermined pixel by, for example, relatively aligning the positions of a plurality of captured images 81 (see FIG. 3) of the subject 30 captured for each wavelength, even if the subject 30 moves slightly when being captured. The spectroscopic camera 100 includes, for example, a measurement unit 10 and a processing unit 20.

[0010] The measurement unit 10 includes an incident optical system 40 into which light from the subject 30 is incident, a bandpass filter (BPF) 50, a spectral filter 60 that spectrally separates the incident light, and an optical sensor 70 that captures the light spectrally separated by the spectral filter 60.

[0011] The incident optical system 40 includes, for example, an autofocus mechanism. The incident optical system 40 is also configured, for example, by a telecentric optical system, and guides the light 31 of the first wavelength number to the spectral filter 60 so that the optical axis and the chief ray are parallel or approximately parallel to each other.

[0012] The spectral filter 60 is, for example, a wavelength selection filter, and a Fabry-Perot type filter capable of changing the transmission wavelength band is used.

[0013] The spectral filter 60 is a tunable interference filter including a pair of substrates 61, 62, a pair of reflective films 63, 64 facing each other, and a gap changer 65 that can change the gap dimension between these reflective films 63, 64. The gap changer 65 is configured by, for example, an electrostatic actuator. The tunable interference filter is also called an etalon. The spectral filter 60 is placed on the optical path of light incident on the optical sensor 70.

[0014] The spectral filter 60 changes the gap dimension of the reflective films 63 and 64 by changing the voltage applied to the gap change unit 65 under the control of the control unit 21 that constitutes the processing unit 20, and changes the output wavelength λi (i=1, 2,..., N), which is the wavelength of the light that passes through the reflective films 63 and 64.

[0015] The optical sensor 70 is, for example, a CCD (Charge Coupled Device), and is an imaging device that photoelectrically converts the light 32 of the second wavelength that has passed through the spectral filter 60 to obtain an electrical signal representing the subject 30.

[0016] The spectroscopic camera 100 sequentially receives instructions for a plurality of measurement bands (multi-bands) from the control unit 21 at the spectroscopic filter 60, thereby sequentially changing the transmission wavelength range of the spectroscopic filter 60. In this way, the spectroscopic camera 100 captures an image of the subject 30 with sensitivity to a plurality of wavelength bands.

[0017] The processing unit 20 includes a control unit 21 , a storage unit 22 , a calculation processing unit 23 , a feature point extraction unit 24 , and a position adjustment unit 25 .

[0018] The control unit 21 is configured to include one or more processors, and performs overall control of the operation of the spectroscopic camera 100 by operating according to a control program stored in the storage unit 22, for example.

[0019] The storage unit 22 is configured with memories such as RAM (Random Access Memory) and ROM (Read Only Memory). The RAM is used for temporary storage of various data, and the ROM stores control programs and control data for controlling the operation of the spectroscopic camera 100. The storage unit 22 also stores captured images 81 captured by the optical sensor 70 for each transmitted wavelength.

[0020] The arithmetic processing unit 23 executes various processes using the data and parameters stored in the storage unit 22. The arithmetic processing unit 23 also performs various arithmetic processes based on information of the captured image 81.

[0021] The feature point extraction unit 24 extracts common feature points among the captured images 81. Specifically, at least three feature points corresponding to the periphery of the captured subject 30, that is, corner points, are extracted.

[0022] The positioning unit 25 uses the feature points of the captured images 81 to match the coordinates of the feature points between all the captured images, that is, performs image correction.

[0023] Next, a spectroscopic measurement method using the spectroscopic camera 100 of the first embodiment will be described with reference to FIGS.

[0024] 2, in step S11, the control unit 21 causes the optical sensor 70 to capture an image of the subject 30 for each wavelength. Specifically, for example, the optical sensor 70 captures an image of a white tile and performs calibration. Thereafter, the optical sensor 70 captures an image of the subject 30 for each set wavelength to obtain a captured image 81, as shown in FIG.

[0025] Note that camera shake in a single captured image 81 can be corrected by, for example, camera shake correction using a known gyro. For example, if correction is not possible, it is preferable to notify the user of an error and have them retake the image. As a result, a clear captured image 81 can be obtained. However, camera shake and other factors can cause deviations in relative position or angle between multiple captured images.

[0026] In step S12, the control unit 21 causes the feature point extraction unit 24 to extract feature points of the captured image 81. Specifically, as shown in FIG. 3, the feature point extraction unit 24 extracts corner points (1, 2, 3, 4) that are feature points of the multiple captured images 81 (81a1, 81a2, 81a3, 81an).

[0027] In step S13, the control unit 21 instructs the positioning unit 25 to align the positions of the other captured images 81a2, 81a3, and 81an using a captured image 81a1 as a first captured image captured first (i.e., the first image) at a first wavelength (i.e., the wavelength measured first). Specifically, the positioning unit 25 causes the calculation processing unit 23 to perform image correction (i.e., projection correction) so that the coordinates of each corner point of the second captured image 81a2 match the coordinates of each corner point of the first captured image 81a1 as a reference, as shown in FIG.

[0028] As a result, for example, the second captured image 81a2, which may have a trapezoidal shape, is corrected to a square captured image 81b2, the same as the first captured image 81a1, as shown in Fig. 4. Similarly, the third captured image 81a3 and the nth captured image 81an (at the nth wavelength, which is the nth wavelength to be measured) are also corrected based on the coordinates of each corner point of the first captured image 81a1. This results in corrected captured images 81b3 and 81bn.

[0029] By using such a spectroscopic measurement method, the feature points of the first captured image 81a1 are used as a reference and the relative positions of the other captured images 81a2, 81a3, and 81an are adjusted (i.e., projectively transformed) so that they coincide with each other. Therefore, even if the image is distorted or the angle is shifted slightly due to camera shake or the like, it can be restored to a correct image. In other words, all captured images 81 are captured facing the same direction. Therefore, a highly accurate spectroscopic spectrum can be obtained at a specified pixel. In other words, an accurate spectroscopic spectrum can be acquired.

[0030] As described above, the spectroscopic measurement method of this embodiment includes a step of capturing an image of the subject 30 for each wavelength to be measured, a step of extracting at least three feature points corresponding to the periphery of the subject 30 for each of all captured images 81 captured in the capturing step, and a step of performing image correction using the feature points of each captured image 81 to match the coordinates of each feature point between all captured images 81.

[0031] According to this method, the coordinates of all captured images 81 are matched based on the feature points, i.e., the relative positions are adjusted. Therefore, even if the subject 30 moves slightly when capturing an image, it is possible to obtain a highly accurate optical spectrum at a specified pixel, thereby improving the analytical accuracy of the captured images 81.

[0032] In the spectroscopic measurement method of this embodiment, the image correction step preferably uses the first captured image 81a1 as a reference and performs image correction so that feature points of the n-th captured image 81an match the feature points of the first captured image 81a1. According to this method, the feature points of the first captured image 81a1 are used as a reference and the relative positions of the other captured images 81 are adjusted so that they match, making it possible to obtain a highly accurate spectroscopic spectrum at a specified pixel.

[0033] The spectroscopic camera 100 of this embodiment also includes a spectral filter 60, an optical sensor 70, an incident optical system 40, a control unit 21 that controls the transmission wavelength of the spectral filter 60, a memory unit 22 that stores captured images 81 captured by the optical sensor 70 for each transmission wavelength, a calculation processing unit 23 that performs calculation processing based on the captured images 81, a feature point extraction unit 24 that extracts at least three feature points corresponding to the periphery of the subject 30 for each captured image 81, and a position alignment unit 25 that uses the feature points of each captured image 81 to match the coordinates of each feature point between all captured images 81.

[0034] According to this configuration, the coordinates of all captured images 81 are matched, i.e., the relative positions are adjusted, based on the feature points. Therefore, even if the subject 30 moves slightly when capturing an image, it is possible to obtain a highly accurate optical spectrum at a specified pixel, thereby improving the analytical accuracy of the captured images 81.

[0035] Next, a spectroscopic measurement method according to a second embodiment will be described with reference to FIGS.

[0036] The spectroscopic measurement method of the second embodiment differs from the first embodiment in that the nth captured image 181an is used as a reference and the captured image 181a(n+1) (i.e., the next consecutive captured image) is matched as the (n+1)th captured image measured at the next wavelength, that is, the (n+1)th wavelength. The other configurations are generally similar. Therefore, in the second embodiment, differences from the first embodiment will be described in detail, and descriptions of other overlapping portions will be omitted as appropriate.

[0037] As shown in FIG. 5, steps S21 and S22 are similar to steps S11 and S12 in the first embodiment.

[0038] In step S23, the control unit 21 instructs the positioning unit 25 to align the position of the next captured image 181a(n+1) with the nth captured image 181an as a reference. Specifically, the positioning unit 25 causes the calculation processing unit 23 to perform image correction so that the coordinates of each corner point of the next captured image 181a(n+1) match with the coordinates of each corner point of the nth captured image 181an as a reference.

[0039] 6, the coordinates of each corner point of the second captured image 181a2 are matched with the coordinates of each corner point of the first captured image 181a1 as a reference, thereby correcting the second captured image 181a2 to a captured image 181b2.

[0040] 7, the coordinates of each corner point of the third captured image 181a3 are matched with the coordinates of each corner point of the corrected second captured image 181b2 as a reference, thereby correcting the third captured image 181a3 to the captured image 181b3.

[0041] 8, the coordinates of each corner point of the (n+1)th captured image 181a(n+1) are matched with the coordinates of each corner point of the corrected nth captured image 181bn as a reference, thereby correcting the (n+1)th captured image 181a(n+1) to the captured image 181b(n+1).

[0042] By using such a spectroscopic measurement method, the feature points of the captured image 181an captured at the nth frame are used as a reference, and the relative positions are adjusted (i.e., projective transformation is performed) so that the feature points of the next captured image 181a(n+1) coincide with each other. Therefore, even if the image is distorted to some extent due to camera shake or the like, it can be restored to a correct image. In other words, all captured images 181 are captured facing the same direction. Therefore, a highly accurate spectroscopic spectrum can be obtained at a predetermined pixel. In other words, an accurate spectroscopic spectrum can be acquired.

[0043] As described above, in the spectroscopic measurement method of this embodiment, the image correction step preferably uses the nth captured image 81an as a reference and performs image correction such that feature points of the (n+1)th captured image 81a(n+1) coincide with feature points of the nth captured image 81an. According to this method, the relative positions are adjusted so that feature points of the (n+1)th captured image 81a(n+1) captured next coincide with feature points of the nth captured image 81an, thereby making it possible to obtain a highly accurate spectroscopic spectrum at a predetermined pixel.

[0044] Modifications of the above-described embodiment will now be described.

[0045] Correction of the captured image 81 is not limited to the above-described first embodiment, in which the other captured images 81a are corrected based on the first captured image 81a1, or the second embodiment, in which correction is made between two consecutive captured images 81an and 81a(n+1), but the captured image 81 may be corrected as follows.

[0046] A modified spectroscopic measurement method may, for example, select an image 81 with the most average angle deviation (referred to as an average image) from a plurality of images 81 based on information from a gyroscope or the like, and relatively correct the remaining images 81 based on the selected average image 81.

[0047] This may increase the amount of calculation required for image correction, but it reduces the relative correction of the captured image 81. Therefore, it is possible to improve the accuracy of image correction.

[0048] As described above, in the spectroscopic measurement method of the modified example, it is preferable to acquire feature points of all captured images 81 captured at all wavelengths, determine an average captured image 81 with the most average feature points of all captured images 81, and correct the remaining captured images 81 using the average captured image 81 as a reference. According to this method, the feature points of the most average captured image 81 are used as a reference and the relative positions of the feature points of the other captured images 81 are adjusted so that they coincide, thereby obtaining a highly accurate spectroscopic spectrum for a specified pixel. Furthermore, since the relative positions of the other captured images are adjusted from the average captured image, the correction accuracy can be improved.

[0049] Furthermore, as described above, four corner points are given as feature points, but this is not limiting, and for example, three points may be used as feature points for image correction. With at least three points, a surface covering the subject 30 can be formed, making it possible to perform image correction. [Explanation of symbols]

[0050] 10...measurement unit, 20...processing unit, 21...control unit, 22...memory unit, 23...arithmetic processing unit, 24...feature point extraction unit, 25...alignment unit, 30...subject, 31...light of first wavelength number, 32...light of second wavelength number, 40...incident optical system, 50...bandpass filter, 60...spectral filter, 61, 62...pair of substrates, 63, 64...pair of reflective films, 65...gap changing unit, 70...optical sensor, 81, 81a, 81b...captured image, 100...spectral camera, 181, 181a, 181b...captured image.

Claims

1. capturing an image of an object for each wavelength to be measured; extracting at least three feature points corresponding to the periphery of the subject for each of all the captured images; performing image correction using the feature points of each of the captured images to make coordinates of each feature point consistent among all of the captured images; and The image correction step is a spectroscopic measurement method in which, when a first captured image captured at a first wavelength is used as a reference, the nth captured image is captured at an nth wavelength different from the first wavelength, which is the nth wavelength to be measured, among all the captured images for each wavelength captured in the imaging step, and image correction is performed so that feature points of the nth captured image match feature points of the first captured image.

2. 2. The spectroscopic measurement method according to claim 1, The image correction step uses the nth captured image captured at the nth wavelength as a reference, and performs image correction so that feature points of the (n+1)th captured image captured at the (n+1)th wavelength, which is the wavelength next to the nth wavelength, match the feature points of the nth captured image.

3. A spectral filter; A light sensor and an incident optical system; a control unit that controls the transmission wavelength of the spectral filter; a storage unit that stores images captured by the optical sensor for each of the transmission wavelengths; a calculation processing unit that performs calculation processing based on the captured image; a feature point extraction unit that extracts at least three feature points corresponding to the periphery of a subject from each of the captured images; a position matching unit that uses the feature points of each of the captured images to match coordinates of each feature point among all of the captured images; Equipped with The alignment unit is a spectroscopic camera that is configured to match feature points of an n-th captured image captured at an n-th wavelength different from the first wavelength, which is an n-th wavelength to be measured when a first captured image captured at a first wavelength is used as a reference, among all the captured images captured for each of the transmitted wavelengths, with feature points of the first captured image.

Images