Spectroscopic camera and spectroscopic measurement method

Abstract

To provide a spectroscopic camera and a spectroscopic measuring method that can determine a foreign substance.SOLUTION: A spectroscopic camera comprises a spectroscopic filter, an optical sensor, and an incident optical system, and the camera comprises: a control unit that controls the transmission wavelength of the spectroscopic filter; a storage unit that stores information on an image picked up by the optical sensor for every transmission wavelength; an arithmetic processing unit that performs arithmetic processing based on the image information; and a determination unit that, based on a result of the arithmetic processing, determines a pixel with a wavelength region in which the absolute value of a differential spectrum is equal to or more than a constant threshold as a foreign substance pixel, excluding normal pixels having a luminance value equal to or more than a predetermined threshold in all measured wavelengths.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a spectroscopic camera and a spectroscopic measurement method. [Background technology]

[0002] Patent Document 1 discloses a method for determining whether a lens is dirty by creating an image using two cameras and comparing the background image with the current image. Patent Document 2 discloses a method for detecting immobile areas in a captured image and determining whether a foreign object is present. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-130549 [Patent Document 2] Japanese Patent Application Laid-Open No. 2012-38048 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the method of Patent Document 1 requires two cameras, which makes it difficult to miniaturize the device and limits the scope of use. The method of Patent Document 2 has the problem that it is difficult to distinguish between a foreign object on the optical path, such as a lens, and a dead pixel (specifically, a pixel that has lost sensitivity). In other words, a compact method that can distinguish foreign objects is needed. [Means for solving the problem]

[0005] The spectroscopic camera is a spectroscopic camera equipped with a spectral filter, an optical sensor, and an incident optical system, and is also equipped with: a control unit that controls the transmission wavelength of the spectral filter; a memory unit that stores image information captured by the optical sensor for each transmission wavelength; an arithmetic processing unit that performs arithmetic processing based on the image information; and a determination unit that, based on the results of the arithmetic processing, determines as foreign matter pixels any pixel whose absolute value of the differential spectrum falls within a wavelength range equal to or greater than a certain threshold, excluding normal pixels whose luminance values ​​are equal to or greater than a predetermined threshold at all measured wavelengths.

[0006] The spectroscopic measurement method is a spectroscopic measurement method using a spectroscopic camera that includes a spectral filter, an optical sensor, an incident optical system, a control unit that controls the transmission wavelength of the spectral filter, a memory unit that stores image information captured by the optical sensor for each of the transmission wavelengths, a calculation processing unit that performs calculation processing based on the image information, and a determination unit that determines whether or not a foreign matter pixel is present based on the result of the calculation processing, and determines, as the foreign matter pixel, a pixel whose absolute value of the differential spectrum exists in a wavelength range that is equal to or greater than a certain threshold, excluding normal pixels that have brightness values ​​equal to or greater than a predetermined threshold at all measurement wavelengths. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of a spectroscopic camera. [Figure 2] 3 is a flowchart showing a spectroscopic measurement method according to the first embodiment. [Figure 3] 10 is a graph illustrating a part of image processing. [Figure 4] 10 is a graph illustrating a part of image processing. [Figure 5] 10 is a graph illustrating a part of image processing. [Figure 6] 10 is a graph illustrating a method for determining foreign matter. [Figure 7] 10 is a graph illustrating a method for determining foreign matter. [Figure 8] FIG. 4 is a diagram showing an example of a display on a display unit. [Figure 9] 10 is a flowchart showing a spectroscopic measurement method according to a second embodiment. [Figure 10] FIG. 10 is a diagram illustrating a processing method according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0008] The configuration of the spectroscopic camera 100 will be described with reference to FIG.

[0009] 1, the spectroscopic camera 100 is used to determine whether a pixel of the measurement target 30 is a pixel where the measurement target 30 is normally imaged (hereinafter referred to as a normal pixel) or a pixel where a foreign substance 200 (see FIG. 10) adhering to a lens or the like is imaged (hereinafter referred to as a foreign substance pixel), when an image of the measurement target 30 is captured. The spectroscopic camera 100 includes, for example, a measurement unit 10, a processing unit 20, and a display unit 25.

[0010] The measurement unit 10 includes an incident optical system 40 into which light from the object to be measured 30 is incident, a bandpass filter (BPF) 50, a spectral filter 60 that spectrally separates the incident light, and an optical sensor 70 that captures the light spectrally separated by the spectral filter 60.

[0011] The incident optical system 40 includes, for example, an autofocus mechanism. The incident optical system 40 is also configured, for example, by a telecentric optical system, and guides the first wavelength light 31 to the spectral filter 60 so that the optical axis and the chief ray are parallel or approximately parallel.

[0012] The spectral filter 60 is, for example, a wavelength selection filter, and a Fabry-Perot type filter capable of changing the transmission wavelength band is used.

[0013] The spectral filter 60 is a tunable interference filter including a pair of substrates 61, 62, a pair of reflective films 63, 64 facing each other, and a gap changer 65 that can change the gap dimension between these reflective films 63, 64. The gap changer 65 is configured by, for example, an electrostatic actuator. The tunable interference filter is also called an etalon. The spectral filter 60 is placed on the optical path of light incident on the optical sensor 70.

[0014] The spectral filter 60 changes the gap dimension of the reflective films 63 and 64 by changing the voltage applied to the gap change unit 65 under the control of the control unit 21 that constitutes the processing unit 20, and changes the output wavelength λi (i=1, 2,..., N), which is the wavelength of the light that passes through the reflective films 63 and 64.

[0015] The optical sensor 70 is, for example, a CCD (Charge Coupled Device), and is an imaging device that photoelectrically converts the light 32 of the second wavelength that has passed through the spectral filter 60 to obtain an electrical signal representing the object 30 to be measured.

[0016] The spectroscopic camera 100 sequentially receives instructions for a plurality of measurement bands (multi-bands) from the control unit 21 at the spectroscopic filter 60, thereby sequentially changing the transmission wavelength range of the spectroscopic filter 60. In this way, the spectroscopic camera 100 captures an image of the measurement target 30 with sensitivity to a plurality of wavelength bands.

[0017] The processing unit 20 includes a control unit 21, a storage unit 22, an arithmetic processing unit 23, and a determination unit 24.

[0018] The control unit 21 is configured to include one or more processors, and performs overall control of the operation of the spectroscopic camera 100 by operating according to a control program stored in the storage unit 22, for example.

[0019] The storage unit 22 is configured with memories such as RAM (Random Access Memory) and ROM (Read Only Memory). The RAM is used for temporary storage of various data, and the ROM stores control programs and control data for controlling the operation of the spectroscopic camera 100. The storage unit 22 also stores image information captured by the optical sensor 70 for each transmitted wavelength.

[0020] The arithmetic processing unit 23 executes various processes using the data and parameters stored in the storage unit 22. The arithmetic processing unit 23 also performs various arithmetic processes based on image information.

[0021] The determination unit 24 determines whether a pixel is normal or a foreign substance pixel from the spectral image of each pixel processed by the calculation processing unit 23. Specifically, based on the results of the calculation processing, the determination unit 24 excludes normal pixels that have brightness values ​​equal to or greater than a predetermined threshold at all measured wavelengths, and determines pixels whose absolute values ​​of the differential spectrum fall within a wavelength range equal to or greater than a certain threshold as foreign substance pixels.

[0022] The display unit 25 is for displaying various information on the screen, and includes, for example, a liquid crystal display.

[0023] Next, a spectroscopic measurement method using the spectroscopic camera 100 of the first embodiment will be described with reference to Figures 2 to 8. The spectroscopic measurement method of the first embodiment determines whether a pixel is a normal pixel or whether a foreign pixel is included when calibrating the spectroscopic camera 100.

[0024] 2, in step S11, the control unit 21 causes the optical sensor 70 to capture an image of the white tile. Specifically, the white tile is a reference measurement object and is considered to have 100% reflectance over the entire wavelength range. Thereafter, calibration is performed based on the image information obtained by capturing an image of the white tile.

[0025] In step S12, the processing unit 20 acquires a raw image. Specifically, the control unit 21 constituting the processing unit 20 causes the calculation processing unit 23 to perform calculation processing based on the image information, thereby acquiring a spectrum such as the graph shown in Fig. 3. In the graph shown in Fig. 3, the horizontal axis two-dimensionally represents the pixel positions (X, Y) of the optical sensor 70. The vertical axis represents the luminance value corresponding to the pixel position, with the luminance value increasing upward.

[0026] The raw image can be obtained based on, for example, D_raw(λn,X,Y). D is the luminance value. λn is the wavelength. X is the horizontal pixel position. Y is the vertical pixel position. The spectrum shown in FIG. 3 is based on a white tile, so the luminance value is essentially flat. However, some distortion occurs depending on the pixel position due to, for example, the optical system. Furthermore, the flat luminance value may have a low luminance value in some pixels due to, for example, a foreign substance 200. It is then determined whether this low luminance value is due to the foreign substance 200.

[0027] In step S13, the control unit 21 causes the calculation processing unit 23 to perform flattening correction. Specifically, the luminance value is corrected to be flat, as shown in Fig. 4. The corrected luminance value D1 can be obtained based on D1(λn, X, Y).

[0028] Examples of methods for flattening correction include various digital filters such as a moving average filter or a median (center value) filter, and fitting using a specific function such as a curved surface function. In this case, it is desirable to adjust the processing parameters so as to preserve the drop in brightness value in the dark area that is thought to correspond to the foreign substance 200.

[0029] In step S14, the control unit 21 causes the calculation processing unit 23 to calculate the spectrum of all pixels. The spectrum of all pixels can be obtained based on (Xn, Yn), (λn, D). Specifically, the spectrum of the graph shown in FIG. 5 is calculated.

[0030] 5, the horizontal axis represents wavelength and the vertical axis represents luminance value. Specifically, the corrected luminance values ​​D1 described above were rearranged, and the spectrum was calculated for all pixels.

[0031] In step S15, the control unit 21 causes the determination unit 24 to perform a process of determining whether or not a foreign pixel is included. Specifically, the determination is performed using the method of determination example 1 shown in Fig. 6 and the method of determination example 2 shown in Fig. 7.

[0032] First, the method of determination example 1 shown in Fig. 6 will be described. In the graph shown in Fig. 6, the horizontal axis indicates wavelength λ, and the vertical axis indicates brightness value (D brightness value in Fig. 6). Note that the brightness value on the vertical axis increases (i.e., becomes brighter) as it goes up. Also, each drawn line corresponds to a pixel.

[0033] Normal pixels have a brightness value equal to or greater than a predetermined threshold T1, and exhibit a waveform that is nearly flat across the entire wavelength range. Dead pixels (specifically, pixels that are not functioning as sensors) have a brightness value less than a predetermined threshold T2, and exhibit a waveform that is nearly flat across the entire wavelength range. The predetermined threshold T2 is, for example, a brightness value close to 0.

[0034] On the other hand, foreign matter pixels do not have a flat waveform overall, but have wavelength dependency in that certain wavelengths become darker, although this differs depending on the type of foreign matter 200. Therefore, in judgment example 1, the judgment unit 24 judges as foreign matter pixels all pixels excluding normal pixels having a value equal to or greater than a predetermined threshold T1 and dead pixels having a value less than a predetermined threshold T2.

[0035] Next, a method of judgment example 2 shown in Fig. 7 will be described. In the graph shown in Fig. 7, the horizontal axis represents wavelength λ, and the vertical axis represents the differential value of luminance (dD / dλ luminance differential value in Fig. 7). In judgment example 2, attention is paid to the fact that the spectrum of a foreign substance pixel is not flat, and judgment is made based on the absolute value of the differential spectrum (shown as +T3, -T3 in Fig. 7). The differential value is the slope.

[0036] Normal pixels and dead pixels have almost no slope with respect to wavelength λ, and therefore appear in the area close to 0. In other words, if a pixel falls between +T3 and -T3, it is determined to be a dead pixel. Note that in Figure 7, the normal pixels determined in Determination Example 1 are excluded, and foreign substance pixels and dead pixels are determined.

[0037] The foreign substance 200 has a characteristic spectrum, and therefore a portion that protrudes from the range of absolute values ​​appears. In other words, if the absolute value of the differential spectrum falls outside the range of +T3 and -T3, that is, if there is a wavelength range equal to or greater than a certain threshold, it is determined to be a foreign substance pixel.

[0038] In step S16, the control unit 21 causes the display unit 25 to display a message urging the user to perform cleaning. Specifically, as shown in FIG. 8, when a foreign pixel is detected, the display unit 25 displays a message urging the user to perform cleaning, such as "Please clean!" After this, the user cleans the incident optical system 40, such as the lens. Note that the cleaning is not limited to the incident optical system 40, and the inside of the optical sensor 70 may also be cleaned, for example.

[0039] As described above, the spectroscopic camera 100 of this embodiment is a spectroscopic camera 100 equipped with the spectral filter 60, the optical sensor 70, and the incident optical system 40, and further equipped with: a control unit 21 that controls the transmission wavelength of the spectral filter 60; a memory unit 22 that stores image information captured by the optical sensor 70 for each transmission wavelength; an arithmetic processing unit 23 that performs arithmetic processing based on the image information; and a determination unit 24 that, based on the results of the arithmetic processing, determines, as foreign matter pixels, pixels whose absolute values ​​of their differential spectra exist in a wavelength range where the absolute values ​​are equal to or greater than a certain threshold, excluding normal pixels whose luminance values ​​are equal to or greater than a predetermined threshold at all measured wavelengths.

[0040] According to this configuration, the transmission wavelength of the spectral filter 60 is controlled, image information captured for each transmission wavelength is calculated, and the determining unit 24 determines whether a pixel is a foreign substance pixel based on the above conditions, making it possible to eliminate pixel information of dead pixels and identify the foreign substance 200 on the optical path. Furthermore, by using the spectral filter 60 or the like, it is possible to provide a compact spectral camera 100 that can determine the foreign substance 200.

[0041] In addition, the user does not have to perform unnecessary cleaning due to false detection of dead pixels. Cleaning also makes it possible to maintain high image quality. Furthermore, the small, space-saving spectroscopic camera 100 can be used in a wider range of applications.

[0042] Furthermore, in the spectroscopic camera 100 of this embodiment, the determination unit 24 preferably determines, as a dead pixel, any pixel whose absolute value of the differential spectrum is less than a certain threshold value across the entire wavelength range, excluding normal pixels, and the control unit 21 feeds back the result of determining that a pixel is a dead pixel to the arithmetic processing unit 23. With this configuration, the result of determining that a pixel is a dead pixel is fed back to the arithmetic processing unit 23, so that the foreign substance 200 can be determined without including image information about the dead pixel. This improves the accuracy of determining the foreign substance 200.

[0043] Furthermore, the spectroscopic camera 100 of this embodiment preferably includes a display unit 25 that displays the determination result by the determination unit 24, and when the determination unit 24 determines that a pixel is a foreign substance, the control unit 21 preferably causes the display unit 25 to display a message urging the user to clean the incident optical system 40. According to this configuration, when a foreign substance pixel is determined to exist, the display unit 25 displays a message urging the user to clean the pixel, allowing the user to remove the foreign substance 200 at the timing indicated by the message.

[0044] Furthermore, the spectroscopic measurement method of this embodiment is a spectroscopic measurement method using a spectroscopic camera 100 that includes a spectral filter 60, an optical sensor 70, an incident optical system 40, a control unit 21 that controls the transmission wavelength of the spectral filter 60, a memory unit 22 that stores image information captured by the optical sensor 70 for each transmission wavelength, a calculation processing unit 23 that performs calculation processing based on the image information, and a determination unit 24 that determines whether or not a foreign object pixel is present based on the result of the calculation processing, and determines as a foreign object pixel any pixel whose absolute value of the differential spectrum exists in a wavelength range that is equal to or greater than a certain threshold, excluding normal pixels that have brightness values ​​equal to or greater than a predetermined threshold at all measurement wavelengths.

[0045] According to this method, the transmission wavelength of the spectral filter 60 is controlled, image information captured for each transmission wavelength is calculated, and the determining unit 24 determines whether a pixel is a foreign substance pixel based on the above conditions. This makes it possible to eliminate pixel information about dead pixels and identify foreign substances 200 on the optical path.

[0046] Next, a spectroscopic measurement method according to the second embodiment will be described with reference to FIG.

[0047] 9, the spectroscopic measurement method of the second embodiment differs from the first embodiment in the part where foreign substance pixels are determined when inspecting multiple items, for example, during mass production. The other configurations are generally the same. Therefore, in the second embodiment, the parts that differ from the first embodiment will be described in detail, and the description of other overlapping parts will be omitted as appropriate.

[0048] In the spectroscopic measurement method of the second embodiment, for example, a spectroscopic camera 100 is used to compare pixel spectra of multiple items P (see FIG. 10) to perform foreign substance inspection in parallel with item inspection. Specifically, for example, the spectroscopic camera 100 is incorporated into an inspection system. First, as shown in FIG. 9, in step S21, the control unit 21 causes the optical sensor 70 to capture an image of the item P. Specifically, for example, an image is captured when multiple items P are being conveyed. The item P is, for example, fruit (see FIG. 10).

[0049] The image processing in steps S22 to S24 is the same as that in steps S12 to S14 in the first embodiment.

[0050] In step S25, the control unit 21 causes the determination unit 24 to determine whether or not the pixel spectra of the multiple items P completely match. The pixel spectra are, for example, the spectra shown in the graph of Fig. 5. If the pixel spectra match for the multiple items P, the process proceeds to step S26. If they do not match, the process proceeds to step S21.

[0051] In step S26, the control unit 21 causes the determination unit 24 to perform a process of determining whether or not a foreign substance pixel is included. Specifically, for multiple products P, pixels whose pixel spectra completely match are determined to be foreign substance pixels or dead pixels, and foreign substance pixels are distinguished from dead pixels based on the presence or absence of feature points in the differential spectrum, as in determination example 2 shown in Figure 7. The method of determination example 2 is similar to that described above.

[0052] In step S27, control unit 21 causes display unit 25 to display a message urging the user to perform cleaning. Specifically, as described above, the message is displayed as shown in Fig. 8. After this, the user cleans incident optical system 40 such as the lens.

[0053] As described above, the spectroscopic camera 100 of the second embodiment compares pixel spectra of multiple products P, and therefore can identify foreign matter pixels even without a reference measurement object such as a white tile. Furthermore, foreign matter inspection can be performed in parallel with product inspection during mass production.

[0054] Modifications of the above-described embodiment will now be described.

[0055] In the above-described determination examples 1 and 2, pixels determined to be dead pixels may be excluded from the effective imaging area 71 of the optical sensor 70. That is, as shown in FIG. 10 , the range of the effective imaging area 71 is changed to the range of a slightly smaller effective imaging area 72.

[0056] As described above, in the modified spectroscopic camera 100, it is preferable that the arithmetic processing unit 23 excludes pixels determined to be dead pixels from the effective imaging area 71 of the optical sensor 70. According to this configuration, since pixels determined to be dead pixels are excluded from the effective imaging area 71, the amount of calculation in the arithmetic processing unit 23 can be reduced when performing arithmetic processing again.

[0057] Furthermore, in the spectroscopic measurement method of this embodiment, it is preferable to determine as dead pixels pixels whose absolute values ​​of differential spectra are less than a certain threshold value across the entire wavelength range, excluding normal pixels, and to remove pixels determined to be dead pixels from the effective imaging area 71 of the optical sensor 70. According to this method, since pixels determined to be dead pixels are removed from the effective imaging area 71, the amount of calculation required when performing calculation processing again can be reduced.

[0058] As described above, the display prompting cleaning is not limited to being displayed on the display unit 25 provided on the spectroscopic camera 100, but may also be displayed on the display unit 81 of a terminal 80 such as a smartphone wirelessly connected to the spectroscopic camera 100 (see Figure 1).

[0059] As described above, in the spectroscopic camera 100 of the modified example, when the determining unit 24 determines that a pixel is a foreign substance pixel, the control unit 21 preferably causes the display unit 81 of the wirelessly connected terminal 80 to display a message urging the user to clean the incident optical system 40. According to this configuration, when a foreign substance pixel is determined to exist, a message urging the user to clean the pixel is displayed on the display unit 81 of the terminal 80, so that the user can know when to clean the pixel even if the user is located far away from the spectroscopic camera 100. When urging the user to clean the pixel, the pixel determined to be a foreign substance pixel is displayed in a color different from other pixels, so that the user can confirm whether the foreign substance has been removed by cleaning.

[0060] As described above, the spectroscopic measurement method performs both judgment example 1 and judgment example 2, but is not limited to this, and foreign substance pixels may be determined using only judgment example 1 or only judgment example 2.

[0061] In the spectroscopic measurement method of this embodiment, if there is a pixel that is determined to be a dead pixel in Judgment Example 1 and Judgment Example 2, by storing in the memory unit that this pixel is a dead pixel, it is possible to omit the subsequent judgments in Judgment Example 1 and Judgment Example 2 for that dead pixel. [Explanation of symbols]

[0062] 10...measurement unit, 20...processing unit, 21...control unit, 22...memory unit, 23...arithmetic processing unit, 24...determination unit, 25...display unit, 30...measurement object, 31...light of first wavelength, 32...light of second wavelength, 40...incident optical system, 50...bandpass filter, 60...spectral filter, 61, 62...substrate, 63, 64...reflective film, 65...gap changing unit, 70...optical sensor, 71...effective imaging area, 72...effective imaging area, 80...terminal, 81...display unit, 100...spectral camera, 200...foreign matter

Claims

1. A spectroscopic camera comprising a spectroscopic filter, a photosensor, and an incident optical system, The transmission wavelength of the spectral filter is controlled, and a white tile, which is a reference measurement object, is applied to the optical sensor. a control unit that captures an image of the a storage unit for storing image information of the white tile captured by the optical sensor for each of the transmission wavelengths; and, By performing arithmetic processing on the image information of the white tile, each pixel of the optical sensor a calculation processing unit for calculating the spectrum of the white tile; Based on the spectrum of the white tile for each pixel of the optical sensor, Absolute values ​​of the differential spectrum are calculated based on the brightness values ​​of normal pixels that are greater than or equal to a certain threshold. and a determination unit that determines that a pixel in the above wavelength range is a foreign substance pixel.

2. The spectroscopic camera according to claim 1 , The determining unit determines whether the absolute value of the differential spectrum is within a certain threshold value in the entire wavelength range, excluding the normal pixels. Pixels below this value are determined to be dead pixels, The control unit feeds back the result of determining the pixel as a dead pixel to the arithmetic processing unit. A spectroscopic camera.

3. The spectroscopic camera according to claim 2, The arithmetic processing unit determines the pixel determined to be the dead pixel as an effective pixel in the optical sensor. Spectroscopic camera excluded from the imaging area.

4. The spectroscopic camera according to any one of claims 1 to 3, a display unit that displays the determination result by the determination unit, When the determining unit determines that the pixel is a foreign substance pixel, the control unit causes the display unit to A spectroscopic camera that displays a message prompting you to clean the incident optical system.

5. The spectroscopic camera according to any one of claims 1 to 3, When the determining unit determines that the pixel is a foreign substance pixel, the control unit A spectroscopic camera that displays a message on the display of the terminal prompting the user to clean the incident optical system.

6. A spectral filter; A light sensor and an incident optical system; The transmission wavelength of the spectral filter is controlled, and a white tile, which is a reference measurement object, is applied to the optical sensor. a control unit that captures an image of the a storage unit for storing image information of the white tile captured by the optical sensor for each of the transmission wavelengths; and, By performing arithmetic processing on the image information of the white tile, each pixel of the optical sensor a calculation processing unit for calculating the spectrum of the white tile; Based on the spectrum of the white tile for each pixel of the optical sensor, it is determined whether or not there is a foreign object pixel. a determination unit that determines whether A spectroscopic measurement method using a spectroscopic camera comprising: Excluding normal pixels that have brightness values ​​above a predetermined threshold at all measurement wavelengths, the differential spectrum a spectroscopic measurement method in which a pixel in which a wavelength range in which the absolute value of Law.

7. 7. The spectroscopic measurement method according to claim 6, Excluding the normal pixels, pixels whose absolute values ​​of the differential spectrum are less than a certain threshold value in the entire wavelength range are A pixel is determined to be a dead pixel, and the pixel determined to be a dead pixel is detected by the optical sensor. Spectroscopic measurement method, excluding from the effective imaging area.

Images