Control method for a spectroscopic imaging device, spectroscopic imaging device, computer program, control method for a display system, control method for a projector, display system, and projector.

Abstract

To provide a control method for a spectral imaging device, the spectral imaging device, and a computer program with which light in a predetermined wavelength can be efficiently taken out.SOLUTION: A control method for a spectral imaging device comprising an image pick-up device and a spectroscopic element, and the method includes, when the spectral imaging device is in a high accuracy mode, creating, with the spectral imaging device, a first measurement spectrum consisting of N1 wavelengths obtained by picking up an image of an object with different output wavelengths of the spectroscopic element, and when the spectral imaging device is in a high speed mode, creating, with the spectral imaging device, a second measurement spectrum consisting of N2 wavelengths obtained by picking up an image of the object with different output wavelengths of the spectroscopic element. N1 is an integer of 2 or more, and N2 is an integer smaller than N1.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a control method for a spectroscopic imaging device, a spectroscopic imaging device, a computer program, a control method for a display system, a control method for a projector, a display system, and a projector.

Background Art

[0002] For example, Patent Document 1 discloses a technique for imaging an image of a test object and correcting the image based on the imaging result. Further, Patent Document 2 describes that an image projected by a projector is sequentially imaged while switching filters in bands corresponding to a plurality of primary colors, and correction data is calculated based on the imaging result. These techniques make it easier to obtain highly accurate correction data as the number of images taken in different wavelength ranges increases, but the time required for imaging becomes longer. In particular, in the case of a spectroscopic imaging device (also referred to as a spectroscopic camera), it is necessary to ensure a predetermined exposure time to obtain a high signal-to-noise ratio for each pixel of the image sensor, so the time required for image correction tends to be long.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, depending on the usage scenario, there are cases where it is not necessary to always obtain a high signal-to-noise ratio and a simple check is desired, such as when high-precision correction data is not required. That is, a spectroscopic imaging device that combines both high-precision measurement and low-precision measurement is required.

Means for Solving the Problems

[0005] The control method for the spectroscopic imaging device comprises an image sensor and a spectroscopic element. When the spectroscopic imaging device is in a first mode, the spectroscopic imaging device generates a first measurement spectrum consisting of N1 wavelengths obtained by imaging an object with different output wavelengths of the spectroscopic element. When the spectroscopic imaging device is in a second mode, the spectroscopic imaging device generates a second measurement spectrum consisting of N2 wavelengths obtained by imaging the object with different output wavelengths of the spectroscopic element, where N1 is an integer of 2 or more, and N2 is an integer smaller than N1.

[0006] The spectroscopic imaging device comprises an image sensor and a spectroscopic element. When the spectroscopic imaging device is in a first mode, the spectroscopic imaging device generates a first measurement spectrum consisting of N1 wavelengths obtained by imaging an object with different output wavelengths of the spectroscopic element. When the spectroscopic imaging device is in a second mode, the spectroscopic imaging device generates N2 second measurement spectra obtained by imaging the object with different output wavelengths of the spectroscopic element, where N1 is an integer greater than or equal to 2, and N2 is an integer less than N1.

[0007] The computer program is a computer program for identifying an object based on imaging data from a spectroscopic imaging device equipped with an image sensor and a spectroscopic element, and causes the computer to execute at least one of the following: in a first mode of the spectroscopic imaging device, imaging the object with different output wavelengths and generating a first measurement spectrum consisting of N1 wavelengths which are integers of 2 or more; and in a second mode of the spectroscopic imaging device, imaging the object with different output wavelengths and generating a second measurement spectrum consisting of N2 wavelengths which are integers smaller than N1.

[0008] Furthermore, one embodiment for solving the above problem is a control method for a display system including a spectroscopic imaging device equipped with an image sensor and a spectroscopic element, and a projector that projects a projection image based on image data onto a projection surface, wherein when the display system is in a first mode, the spectroscopic imaging device generates N1 first image data by capturing the projection image with different spectral wavelengths of the spectroscopic element, the projector generates first corrected image data by correcting the first image data based on the N1 first image data, and the projector projects a first projection image based on the first corrected image data onto the projection surface, wherein when the display system is in a second mode, the spectroscopic imaging device generates N2 second image data by capturing the projection image with different spectral wavelengths of the spectroscopic element, the projector generates second corrected image data by correcting the first image data based on the N2 second image data, and the projector projects a second projection image based on the second corrected image data onto the projection surface, wherein N1 is an integer of 2 or more, and N2 is an integer smaller than N1.

[0009] In the control method for the display system described above, predetermined information relating to color is measured based on the N1 first imaging data, and the data obtained by correcting the first image data based on the measurement results may be the first corrected image data.

[0010] In the control method for the display system described above, when in the second mode, the predetermined measurement result obtained from N2 second imaging data may be converted into information corresponding to the predetermined measurement result obtained from N1 first imaging data based on the conversion data, and the data obtained by correcting the first image data based on the converted information may be used as the second corrected image data.

[0011] In the control method for the display system described above, the projector may project an image including an OSD menu that includes the selection of the first mode and the second mode.

[0012] In the control method for the display system described above, the spectroscopic element may be a tunable interference filter having a pair of reflective films and a gap changing section that can change the gap dimension of the pair of reflective films, and positioned on the optical path of light incident on the image sensor.

[0013] Another embodiment for solving the above problem is a method for controlling a projector that projects a projection image based on image data onto a projection surface, wherein when the projector is in a first mode, N1 first imaging data are acquired by a spectroscopic imaging device, with the spectral wavelengths of the spectroscopic elements of the spectroscopic imaging device being different; first corrected image data is generated by correcting the first image data based on the N1 first imaging data; and a first projection image based on the first corrected image data is projected onto the projection surface; and when the projector is in a second mode, N2 second imaging data are acquired by a spectroscopic imaging device, with the spectral wavelengths of the spectroscopic elements of the spectroscopic imaging device being different; second corrected image data is generated by correcting the first image data based on the N2 second imaging data; and a second projection image based on the second corrected image data is projected onto the projection surface, wherein N1 is an integer of 2 or more, and N2 is an integer smaller than N1.

[0014] Another embodiment for solving the above problem is a display system including a spectral imaging device equipped with an image sensor and a spectroscopic element, and a projector that projects a projection image based on image data onto a projection surface, wherein when the display system is in a first mode, the spectral imaging device generates N1 first imaging data obtained by imaging the projection image with different spectral wavelengths of the spectroscopic element, the projector generates first corrected image data obtained by correcting the first image data based on the N1 first imaging data, and projects a first projection image based on the first corrected image data onto the projection surface, and when the display system is in a second mode, the spectral imaging device generates N2 second imaging data obtained by imaging the projection image with different spectral wavelengths of the spectroscopic element, the projector generates second corrected image data obtained by correcting the first image data based on the N2 second imaging data, and projects a second projection image based on the second corrected image data onto the projection surface, wherein N1 is an integer of 2 or more, and N2 is an integer smaller than N1.

[0015] Another embodiment for solving the above problem is a projector that projects a projection image based on image data onto a projection surface, wherein when the projector is in a first mode, N1 first imaging data are acquired by a spectroscopic imaging device, with the spectral wavelengths of the spectroscopic elements of the spectroscopic imaging device being different; first corrected image data is generated by correcting the first image data based on the N1 first imaging data; and a first projection image based on the first corrected image data is projected onto the projection surface; and when the projector is in a second mode, N2 second imaging data are acquired by a spectroscopic imaging device, with the spectral wavelengths of the spectroscopic elements of the spectroscopic imaging device being different; second corrected image data is generated by correcting the first image data based on the N2 second imaging data; and a second projection image based on the second corrected image data is projected onto the projection surface, wherein N1 is an integer of 2 or more, and N2 is an integer smaller than N1. [Brief explanation of the drawing]

[0016] [Figure 1]Block diagram showing the configuration of the spectroscopic imaging device according to the first embodiment. [Figure 2] Cross-sectional view showing the configuration of the imaging device. [Figure 3] Diagram showing a specific example of the operation mode. [Figure 4] Flowchart showing the procedure for sharpness determination in the high-precision mode. [Figure 5] Flowchart showing the procedure for sharpness determination in the high-speed mode. [Figure 6] Flowchart showing the procedure for sharpness determination in the optimal mode. [Figure 7] Graph showing the relationship between the wavelength of light and the reflectance when the sharpness is different. [Figure 8] Cross-sectional view showing the configuration of the spectroscopic element of the modification. [Figure 9] Block diagram showing the configuration of the display system according to the second embodiment. [Figure 10] Schematic configuration diagram of the spectroscopic imaging unit. [Figure 11] Diagram showing specific examples of the high-precision mode and the high-speed mode. [Figure 12] Flowchart showing the operation of the display system. [Figure 13] Diagram showing the modified configuration of the display system. [Figure 14] Block diagram showing the configuration of the display system according to the third embodiment. [Figure 15] Diagram showing the modified configuration of the display system.

Embodiments for Carrying Out the Invention

[0017] [First Embodiment] Hereinafter, embodiments will be described with reference to the accompanying drawings. First, the configuration of the spectroscopic imaging device 400A will be described with reference to FIG. 1.

[0018] As shown in Figure 1, the spectroscopic imaging device 400A is used, for example, to determine the freshness of a target object T, which is a green vegetable (e.g., spinach, komatsuna, bell pepper, etc.). The spectroscopic imaging device 400A comprises an imaging device 10, a display device 30, a storage device 40, and a processing device 50.

[0019] The imaging device 10 comprises an incident optical system 11, a spectroscopic element 12, an image sensor 13, and a light source unit 14. The incident optical system 11 includes, for example, an autofocus mechanism. The spectroscopic element 12 is, for example, a wavelength-selective filter, and a Fabry-Perot type filter with a changeable transmission wavelength band is used.

[0020] The image sensor 13 includes a first image sensor and a second image sensor (not shown). The first image sensor is a CCD (Charge Coupled Device), an imaging device that converts light transmitted through the spectroscopic element 12 into electrical signals to obtain an electrical signal representing an object. The second image sensor is, for example, a CCDAFE (Analog Front End), and is for digitizing the detection signal of the first image sensor. The light source unit 14 is for illuminating the object T.

[0021] In the imaging device 10, the spectroscopic element 12 receives instructions for multiple measurement bands (multiband) sequentially from the processing device 50, thereby sequentially changing the transmission wavelength range of the spectroscopic element 12. In this way, the imaging device 10 images the object T with sensitivity across multiple wavelength bands.

[0022] The display device 30 is a device for displaying information on a screen. The storage device 40 is an external device for storing data, such as a hard disk drive.

[0023] The processing device 50 is a device that determines the freshness of an object T by processing imaging data obtained by imaging device 10. The processing device 50 includes a control unit 60 and a processing unit 70 that function as a computer, and a storage unit 80.

[0024] The control unit 60 is configured with one or more processors and, for example, operates according to a control program stored in the memory unit 80 to comprehensively control the operation of the spectrophotometer 400A.

[0025] The processing unit 70 performs various processes by executing a control program as a computer program. The storage unit 80 is configured with memory such as RAM (Random Access Memory) and ROM (Read Only Memory). RAM is used for temporary storage of various data, and ROM stores control programs and control data for controlling the operation of the spectroscopic imaging device 400A.

[0026] The storage unit 80 includes a measurement bandwidth data storage unit 81, a setting data storage unit 82, and a conversion data storage unit 83. Although not shown in the diagram, the storage unit 80 also stores a freshness determination program. The data stored in these storage units 80 will be described in detail later.

[0027] The processing unit 70 executes the freshness determination program stored in the storage unit 80, thereby performing the processing of the measurement band indicator unit 71. The processing unit 70 performs each process using the data and parameters stored in the corresponding storage unit 80.

[0028] The imaging control unit 61 of the control unit 60 causes the imaging device 10 to perform imaging. In this case, the imaging control unit 61 sets the imaging conditions for the first mode (high-precision mode M1), the second mode (high-speed mode M2), and the optimal mode M3 based on the setting data in the setting data storage unit 82.

[0029] As a result of each process, the processing unit 50 acquires the imaging data obtained by the imaging device 10, determines the freshness of the object T from the imaging data, and transmits the determination result to the display device 30 and the storage device 40. As a result, the freshness determination result is displayed on the display device 30 and stored in the storage device 40.

[0030] Next, the configuration of the imaging device 10 will be explained with reference to Figure 2.

[0031] The imaging device 10 includes an incident optical system 11 into which ambient light is incident, a spectroscopic element 12 that spectrally analyzes the incident light, and an image sensor 13 that images the light spectrally analyzed by the spectroscopic element 12. The incident optical system 11 is configured, for example, as a telecentric optical system, and guides the incident light to the spectroscopic element 12 and the image sensor 13 so that the optical axis and the principal ray are parallel or substantially parallel.

[0032] The spectroscopic element 12 uses a tunable interference filter comprising a pair of substrates 14a and 14b, a pair of reflective films 15 and 16 facing each other, and a gap changing section 17 capable of changing the gap dimension of these reflective films 15 and 16. The gap changing section 17 is composed of, for example, an electrostatic actuator. A tunable interference filter is also called an etalon. This spectroscopic element 12 is positioned on the optical path of light incident on the image sensor 13.

[0033] The spectroscopic element 12 changes the gap dimensions of the reflective films 15 and 16 by changing the voltage applied to the gap changing section 17 under the control of the processing device 50, thereby changing the output wavelength λi (i=1,2,··,N), which is the wavelength of light transmitted through the reflective films 15 and 16.

[0034] The image sensor 13 is a device that captures light transmitted through the spectroscopic element 12, and is composed of, for example, a CCD or CMOS sensor. The imaging device 10 sequentially switches the wavelength of light spectrally separated by the spectroscopic element 12 according to the control of the control unit 60, captures the light transmitted through the spectroscopic element 12 with the image sensor 13, and outputs imaging data.

[0035] The imaging data is data output for each pixel constituting the image sensor 13, and it represents the intensity of the light received by that pixel, i.e., the amount of light. The imaging data output by the imaging device 10 is input to the processing device 50. Since the imaging device 10 is a wavelength scanning type, it can obtain imaging data with higher resolution compared to the case of a wavelength dispersive type.

[0036] Next, referring to Figure 3, we will explain the operating modes that define the operating state during measurement. In the following, the term "spectral spectrum" may sometimes be simply referred to as "spectrum."

[0037] As shown in Figure 3, there are three operating modes: high-precision mode M1, high-speed mode M2, and optimal mode M3. High-precision mode M1 is an operating mode that enables high-precision measurement by setting the number of wavelengths of light (output wavelength λi) dispersed by the spectrometer 12 to a relatively large number, N1. In contrast, high-speed mode M2 ​​is an operating mode that shortens the measurement time by setting the number of wavelengths of light (output wavelength λi) dispersed by the spectrometer 12 to a smaller number, N2. Optimal mode M3 is an operating mode that sets the measurement time to approximately midway between high-precision mode M1 and high-speed mode M2 ​​by selectively increasing or decreasing the number of wavelengths of light (output wavelength λi) dispersed by the spectrometer 12.

[0038] Specifically, as shown in Figure 3, the high-precision mode M1 sets the output wavelength λi in 10 nm increments within the range of 400 nm to 700 nm, and the number of measurement wavelengths N1 is 31. The high-precision mode M1 is preferably selected when, for example, you want to measure the concentration of a specific substance from a mixture of various types with high precision, when there are multiple objects T to be measured, when the spectral spectrum of the object T has a steep peak shape, or when the shape of the spectral spectrum of the object T is unknown.

[0039] Furthermore, as shown in Figure 3, the high-speed mode M2 ​​sets the output wavelength λi in 40 nm increments within the range of 400 nm to 680 nm, and the number of measurement wavelengths N2 is 8. High-speed mode M2 ​​is preferably selected when, for example, low-precision measurements often require a short cycle time, when the spectral shape of the object T is smooth, or when the spectral shape of the object T is known and it is sufficient to observe a specific few waveforms.

[0040] Furthermore, as shown in Figure 3, the optimal mode M3 is set, for example, by setting the output wavelength λi in 40 nm increments within the range of 400 nm to 600 nm, resulting in 5 measurement wavelengths, and further setting the output wavelength λi in 10 nm increments within the range of 600 nm to 700 nm, resulting in 11 measurement wavelengths. The optimal mode M3 is preferably selected when, for example, the shape of the spectral spectrum of the object T is known, and it is known which wavelengths should be examined in detail and which wavelengths can be thinned out, or when the measurement is performed first in high-precision mode M1 and the output wavelength λi to be measured is selected based on that information.

[0041] In each operating mode, the exposure time is set to a level that allows the image sensor 13 to achieve a sufficient signal-to-noise ratio, enabling the acquisition of highly accurate imaging data. For example, when the exposure time is set to 60 msec, the measurement time in high-precision mode M1 is 2.07 seconds, in high-speed mode M2 ​​it is 0.53 seconds, and in optimal mode M3 it is 1.06 seconds. Note that N1 is an integer greater than or equal to 2, and N2 is an integer smaller than N1, so the values ​​of N1 and N2 can be changed as appropriate. Furthermore, high-precision mode M1 is an example of the first mode, and high-speed mode M2 ​​is an example of the second mode.

[0042] Next, referring to Figures 4 to 6, we will explain how to determine the freshness of green vegetables according to each operating mode M1, M2, and M3.

[0043] First, referring to Figure 4, we will explain how to determine the freshness of object T in high-precision mode M1. For example, by operating the control panel (not shown) provided on the display device 30, you can select measurement in high-precision mode M1.

[0044] In step S11, a database is generated and saved. Specifically, a database necessary for freshness determination is generated and saved in the storage unit 80 of the processing unit 50. The database referred to here corresponds to the various data stored in the storage unit 80 (see Figure 1) of the processing unit 50.

[0045] This section explains the content and generation method of various data used in the database. The data consists of measurement bandwidth data. Note that the database generation and storage may be performed before the 400A spectrophotometer is shipped from the factory, or it may be omitted when using an object T whose freshness is known. Furthermore, some of the procedures may be performed by an operator.

[0046] Furthermore, the setting data is data that sets the processing conditions for various processes executed by the processing unit 70, and for example, it is data that includes the imaging conditions for each operating mode M1, M2, and M3, including N1 and N2.

[0047] As the transformation data, the transformation matrix M of equation (1) below can be used. p is a vector representing the measurement spectrum (first measurement spectrum) in high-precision mode M1, and in this embodiment it consists of 31 elements. x is a vector representing the measurement spectrum in high-speed mode M2, and in this embodiment it consists of 8 elements. The transformation matrix M is determined to estimate a spectrum consisting of m1 wavelengths from a spectrum consisting of m2 wavelengths. With this transformation data (transformation matrix M), it is possible to estimate the measurement spectrum that would be obtained in high-precision mode M1 from the measurement spectrum (second measurement spectrum) obtained in high-speed mode M2. As for how to derive the transformation matrix M, the method described in Japanese Patent Application Publication No. 2012-242270 can be used. Also, in this embodiment, equation (1) holds true for each pixel. p = Mx ... (1)

[0048] In other embodiments, the spectrum of the object may be estimated using a transformation matrix Mq to approximate the measurement spectrum p obtained in high-precision mode M1 to a measurement spectrum obtained with a more accurate spectrometer (for example, measuring at more than 31 wavelengths). The method for determining the transformation matrix Mq can be the method described in Japanese Patent Application Publication No. 2012-242270. The same applies to high-speed mode M2. In this embodiment, the "measurement spectrum" is a collection of brightness values ​​output by any pixel (or group of pixels) of the image sensor 13, arranged along the wavelength.

[0049] Next, we will explain the measurement bandwidth data. When green vegetables age, the chlorophyll breaks down and their vibrant green color disappears. From this, it can be seen that the freshness of green vegetables can be determined from the amount of chlorophyll. In this embodiment, the freshness of the green vegetables, which are the target object T, is determined by estimating the amount of chlorophyll as a characteristic quantity of the target object T.

[0050] Here, referring to Figure 7, we will explain the relationship between light wavelength and reflectance in green vegetables with varying degrees of freshness.

[0051] As shown in Figure 7, fresh vegetables (or slightly wilted vegetables) absorb light due to chlorophyll at around 700 nm. Therefore, multiple wavelengths within the range of 500 nm to 1100 nm, which include the wavelength at which chlorophyll absorbs light (approximately 700 nm), are stored as measurement band data in order to instruct the imaging device 10 to use them as the measurement band.

[0052] Returning to Figure 4, in step S12, the output wavelength λi, which is the imaging condition, is acquired. Specifically, the control unit 60 acquires the imaging conditions for high-precision mode M1, which are included in the setting data storage unit 82.

[0053] Specifically, for example, 31 measurement bandwidth data points are acquired in 10nm increments from 400nm to 700nm. These measurement bandwidth data points would be, for example, 400nm, 410nm, 420nm, ..., 700nm.

[0054] Note that the above measurement bandwidth data does not necessarily have to be 31 points spaced 10 nm apart; 20 nm spacing is also acceptable. Furthermore, the wavelength range is not limited to 400 nm to 700 nm; it may also be in the range of 350 nm to 1100 nm.

[0055] In step S13, the target object T is imaged. Specifically, the imaging control unit 61 controls the imaging device 10 to change the output wavelength λi according to the acquired imaging conditions and image the target object T.

[0056] In step S14, N1 imaging data are generated. Specifically, N1 imaging data are generated by sequentially changing the output wavelength λi of the spectroscopic element 12. In this embodiment, N1 imaging data are equivalent to 31 imaging data. In this embodiment, the 31 imaging data represent the first spectral spectrum.

[0057] In step S15, freshness is determined. Specifically, the processing device 50 determines the freshness of the object T from the imaging data output from the imaging device 10.

[0058] In step S16, the results are displayed and saved. Specifically, the freshness determination results obtained in step S15 are output to the display device 30 and the storage device 40.

[0059] In this way, by performing the freshness determination of green vegetables (object T) in high-precision mode M1, it becomes possible to perform precise measurements, for example, and even the slightest wilting can be determined to be poor freshness. This allows, for example, to select only fresh vegetables.

[0060] Next, with reference to Figure 5, a method for determining the freshness of object T in high-speed mode M2 ​​will be explained. For example, the operation panel (not shown) on the display device 30 is used to select measurement in high-speed mode M2.

[0061] First, similar to the high-precision mode M1, step S11 generates and saves the database.

[0062] Next, in step S21, the output wavelength λi, which is the imaging condition, is acquired. Specifically, the control unit 60 acquires the imaging conditions for high-speed mode M2, which are stored in the setting data storage unit 82.

[0063] Specifically, for example, eight measurement bandwidth data points are acquired at 40nm intervals from 400nm to 680nm. These measurement bandwidth data points would be, for example, 400nm, 440nm, 480nm, ..., 680nm.

[0064] The above measurement bandwidth data does not necessarily have to be eight points spaced 40 nm apart; it is preferable that the measurement can be performed in a shorter time than in high-precision mode M1, and the measurement interval or wavelength range may be changed.

[0065] In step S22, the target object T is imaged. Specifically, the imaging control unit 61 controls the imaging device 10 to change the output wavelength λi according to the acquired imaging conditions and image the target object T.

[0066] In step S23, N2 imaging data are generated. Specifically, N2 imaging data are generated by sequentially changing the output wavelength λi of the spectroscopic element 12. In this embodiment, N2 imaging data are equivalent to 8 imaging data.

[0067] In step S24, optionally, a spectral spectrum is estimated as a second spectral spectrum. Specifically, the spectral spectrum obtained from N1 imaging data is estimated based on the converted data M stored in the converted data storage unit 83 of the storage unit 80.

[0068] Subsequently, steps S25 and S26 are performed in the same manner as steps S15 and S16 in high-precision mode M1.

[0069] As described above, conversion data is acquired in advance to correlate the spectral spectrum results of colors obtained from N1 imaging data in high-precision mode M1 with the spectral spectrum results of colors obtained from N2 imaging data in high-speed mode M2. Therefore, in high-speed mode M2, the spectral spectrum results of colors obtained from N2 imaging data are converted into information equivalent to the spectral spectrum results of colors obtained from N1 imaging data, based on the conversion data. Then, the input imaging data based on the converted information is used as the corrected imaging data. This makes the deviations in peak wavelengths of each color that occur when estimating the spectral spectrum with a relatively small number of imaging data points within an acceptable range, thereby maintaining sufficient measurement accuracy and enabling appropriate correction.

[0070] Thus, by performing the freshness determination of green vegetables, which are the target object T, in high-speed mode M2, it becomes possible to determine freshness in a shorter time compared to, for example, high-precision mode M1, and the efficiency of measurement can be improved when a rough freshness determination is sufficient.

[0071] Next, with reference to Figure 6, we will explain how to determine freshness in optimal mode M3. For example, optimal mode M3 is selected by operating the control panel located on the display device 30.

[0072] Step S31 determines whether the freshness judgment process is being performed for the first time. If it is the first measurement, the process proceeds to step S32. If it is the second or later measurement, the process proceeds to step S33.

[0073] In step S32, freshness measurement is performed in high-precision mode M1, similar to the flow shown in Figure 4. If it is the second or subsequent measurement, the process in step S32 has already been performed, so the process proceeds to step S33.

[0074] Next, in step S33, the output wavelength λi, which is the imaging condition, is acquired. Specifically, the control unit 60 acquires the imaging conditions for the optimal mode M3, which are stored in the setting data storage unit 82.

[0075] Specifically, for example, five measurement bandwidth data points are acquired in 40nm increments from 400nm to 600nm. These measurement bandwidth data points would be, for example, 400nm, 440nm, ...nm, and 600nm. Furthermore, eleven measurement bandwidth data points are acquired in 10nm increments from 600nm to 700nm. These measurement bandwidth data points would be, for example, 600nm, 610nm, ...nm, and 700nm.

[0076] Note that the measurement bandwidth data is not limited to the above. For example, the spacing of the output wavelengths λi may be varied (in other words, unequal spacing may be used) based on at least one of the first spectral spectrum obtained in high-precision mode M1 and the second spectral spectrum obtained in high-speed mode M2. Here, the spacing of the output wavelengths λi may be varied based on the first spectral spectrum.

[0077] For example, as shown in Figure 7, the measurement may be set to measure in wavelength bands with large changes in reflectivity (in other words, wavelength bands with large fluctuations in the spectral distribution), such as measuring in 10 nm increments in the 500 nm to 600 nm range, or in 10 nm increments in the 700 nm to 800 nm range. Alternatively, in other wavelength bands (in other words, wavelength bands with flat spectral distributions), measurements may be set to 40 nm increments. Furthermore, the interval between wavelengths to be measured is not limited to 40 nm or 10 nm increments. By measuring in this way, measurements can be performed in a shorter time and with suppressed accuracy degradation compared to high-precision mode M1.

[0078] In step S34, the target object T is imaged. Specifically, the imaging control unit 61 controls the imaging device 10 to change the output wavelength λi according to the acquired imaging conditions and image the target object T.

[0079] In step S35, imaging data is generated. Specifically, imaging data is generated by sequentially changing the output wavelength λi of the spectroscopic element 12. In this embodiment, for example, there are 16 pieces of imaging data.

[0080] In step S36, the spectral distribution is estimated based on the above calculation formula. This yields an image of the spectral distribution for each set wavelength (color).

[0081] Subsequently, steps S37 and S38 are performed in the same manner as steps S15 and S16 in high-precision mode M1.

[0082] Thus, by performing the freshness determination of green vegetables (target object T) in the optimal mode M3, for example, compared to the high-precision mode M1, the wavelength range that greatly affects freshness determination is known, allowing for precise measurement of that wavelength range while measuring other parts more coarsely. As a result, freshness determination can be performed in a shorter time compared to the high-precision mode M1, while suppressing a decrease in measurement accuracy. Consequently, the efficiency of measurement can be improved.

[0083] As described above, the control method for the spectroscopic imaging device 400A of this embodiment includes an image sensor 13 and a spectroscopic element 12. When the spectroscopic imaging device 400A is in high-precision mode M1, the spectroscopic imaging device 400A generates a first measurement spectrum consisting of N1 wavelengths obtained by imaging the object T with different output wavelengths λi of the spectroscopic element 12. When the spectroscopic imaging device 400A is in high-speed mode M2, the spectroscopic imaging device 400A generates a second measurement spectrum consisting of N2 wavelengths obtained by imaging the object T with different output wavelengths λi of the spectroscopic element 12, where N1 is an integer of 2 or more, and N2 is an integer smaller than N1.

[0084] According to this method, N1 is an integer greater than or equal to 2, and N2 is an integer less than N1. Therefore, for precise measurements, high-precision mode M1 can be used. On the other hand, for simpler measurements, high-speed mode M2 ​​can be used to measure the object T in a shorter time compared to high-precision mode M1. Thus, it is possible to improve user convenience and efficiency for measurements in various scenarios, and to selectively perform high-precision and low-precision measurements.

[0085] Furthermore, it is preferable to derive, or estimate, the spectrum of the object T based on the first measurement spectrum generated in high-precision mode M1, or the second measurement spectrum generated in high-speed mode M2.

[0086] According to this method, the spectrum of the object T is derived, or estimated, based on the first and second measured spectra. This allows for the deriving of a spectral spectrum in which the peak wavelengths of each color are located at approximately the same wavelengths as the spectral spectra obtained from N1 or N2 imaging data.

[0087] Furthermore, it is preferable to estimate the spectrum of the object T by converting the second measurement spectrum obtained in high-speed mode M2 ​​using conversion data that estimates a spectrum consisting of N1 wavelengths from a spectrum consisting of N2 wavelengths.

[0088] This method allows us to obtain conversion data in advance, making it possible to obtain N1 equivalent image data from N2 image data points that are smaller than N1.

[0089] Furthermore, the spectrum of the object T has a first measurement spectrum obtained in high-precision mode M1 and a second measurement spectrum obtained in high-speed mode M2, and it is preferable to vary the spacing of the output wavelengths λi based on at least one of the first measurement spectrum and the second measurement spectrum.

[0090] This method allows for efficient measurements while suppressing the accuracy of imaging data by selecting wavelength bands that can be thinned out based on the spectral shape. For example, one or two imaging data points can be acquired in wavelength bands with flat spectra, while more detailed imaging data can be acquired in wavelength bands with peaks.

[0091] Furthermore, it is preferable that the spectroscopic element 12 is a tunable interference filter positioned on the optical path of light incident on the image sensor 13, and has a pair of reflective films 15, 16 and a gap changing section 17 that can change the gap dimension between the pair of reflective films 15, 16.

[0092] This method uses a tunable interference filter, which allows for high-resolution measurements and short-duration measurements, compared to, for example, wavelength-dispersive types.

[0093] The spectroscopic imaging device 400A comprises an image sensor 13 and a spectroscopic element 12. When the spectroscopic imaging device 400A is in high-precision mode M1, the spectroscopic imaging device 400A generates a first measurement spectrum consisting of N1 wavelengths obtained by imaging the object T with different output wavelengths λi of the spectroscopic element 12. When the spectroscopic imaging device 400A is in high-speed mode M2, the spectroscopic imaging device 400A generates N2 second measurement spectra obtained by imaging the object T with different output wavelengths λi of the spectroscopic element 12, where N1 is an integer greater than or equal to 2, and N2 is an integer less than N1.

[0094] In this configuration, N1 is an integer greater than or equal to 2, and N2 is an integer less than N1. Therefore, for precise measurements, high-precision mode M1 can be used. On the other hand, for simpler measurements, high-speed mode M2 ​​can be used to measure the object T in a shorter time compared to high-precision mode M1. Thus, it is possible to improve user convenience and efficiency for measurements in various scenarios, and to selectively perform high-precision and low-precision measurements.

[0095] The computer program is a computer program for identifying an object T based on imaging data from a spectroscopic imaging device 400A equipped with an image sensor 13 and a spectroscopic element 12, and causes the computer, acting as a processing unit 70, to execute at least one of the following: a process in which, in high-precision mode M1 of the spectroscopic imaging device 400A, the object T is imaged with different output wavelengths λi and a first measurement spectrum consisting of N1 wavelengths that are integers of 2 or more; and a process in which, in high-speed mode M2 ​​of the spectroscopic imaging device 400A, the object T is imaged with different output wavelengths λi and a second measurement spectrum consisting of N2 wavelengths that are integers smaller than N1.

[0096] According to this computer program, N1 is an integer greater than or equal to 2, and N2 is an integer less than N1. Therefore, for precise measurements, high-precision mode M1 can be used. On the other hand, for simpler measurements, high-speed mode M2 ​​can be used to measure the object T in a shorter time compared to high-precision mode M1. Thus, it is possible to improve user convenience and efficiency for measurements in various scenarios, and to selectively perform high-precision and low-precision measurements.

[0097] The following describes some variations of the above embodiment.

[0098] Note that the spectroscopic element 12 is not limited to the configuration described above, and may have a configuration as shown in Figure 8. Figure 8 is a cross-sectional view showing the structure of a modified spectroscopic element 112. The modified spectroscopic element 112 differs from the spectroscopic element 12 of the above embodiment in that the portion composed of the first substrate 101, the second substrate 102, and the third substrate 103 is.

[0099] As shown in Figure 8, in the modified spectroscopic element 112, as described above, the first substrate 101, the second substrate 102, and the third substrate 103 are bonded together, for example, via a bonding layer 106. A pair of reflective films 104 are arranged on the mutually opposing surfaces of the second substrate 102 and the third substrate 103. An electrostatic actuator 105, capable of changing the gap dimension of the reflective films 104, is arranged on the mutually opposing surfaces of the first substrate 101 and the second substrate 102. Even with such a structure, a spectroscopic element 112 having the same function as the spectroscopic element 12 can be provided.

[0100] Furthermore, while the optimal mode M3 in the above embodiment involved performing a measurement in high-precision mode M1 during the first measurement, then acquiring the imaging conditions for optimal mode M3 and imaging the object T, this is not limited to this. For example, if various information from a measurement performed in high-precision mode M1 has already been obtained, processing may be started from step S33 (see Figure 6) even for the first measurement.

[0101] As described above, the spectrophotometer 400A was used to determine the freshness of objects T such as vegetables, but it is not limited to this. For example, it may be used for color determination of exterior parts made of resin or metal, printed materials, dyed fibers, displays, and other display elements. Furthermore, spectroscopic component analysis can be used to determine the presence or absence of moisture and organic matter, and to calibrate concentrations. Again, it is not limited to these, and can be used for anything that can be identified by spectroscopy. Examples of applications using the spectrophotometer 400A include colorimeters for printers and image quality inspection cameras. Examples of image quality inspection cameras include color inspection and stain and dirt (adhered matter) inspection.

[0102] [Second Embodiment] The following describes an embodiment of the display system with reference to the attached drawings. Figure 9 is a block diagram showing the configuration of a display system 1 having a projector 100. In this display system 1, a configuration equivalent to a spectroscopic imaging device is integrated with the projector 100. The spectroscopic imaging device, also called a spectroscopic camera, is not limited to hardware such as the spectroscopic imaging unit 137 described later, but also includes software and a processor for realizing the operation of the spectroscopic imaging device.

[0103] The projector 100 includes an image projection system that generates image light and projects it onto a screen SC that constitutes the projection surface, an image processing system that electrically processes image data which is the basis of the optical image, and a spectral imaging unit 137 that captures the image light displayed on the screen SC. The projector 100 also includes a control unit 150 that controls the image projection system, the image processing system, and the spectral imaging unit 137.

[0104] [Image projection system] The image projection system comprises a projection unit 110 and a drive unit 120. The projection unit 110 is an example of a display unit that displays an image corresponding to the projected image. The projection unit 110 comprises a light source 111, an optical modulator 113, and an optical unit 117. The drive unit 120 comprises a light source drive circuit 121 and an optical modulator drive circuit 123. The light source drive circuit 121 and the optical modulator drive circuit 123 are connected to a bus 180 and communicate data with other components of the projector 100, which are also connected to the bus 180, via the bus 180. Other components include, for example, the control unit 150 and the image processing unit 143 shown in Figure 9.

[0105] The light source 111 can be a solid-state light source such as an LED (Light Emitting Diode) or a laser light source. Alternatively, a lamp such as a halogen lamp, xenon lamp, or ultra-high pressure mercury lamp can be used as the light source 111. A light source drive circuit 121 is connected to the light source 111. The light source drive circuit 121 supplies drive current or pulses to the light source 111 to turn it on, and stops the supplied drive current or pulses to turn off the light source 111.

[0106] The optical modulator 113 includes an optical modulation element that modulates the light emitted by the light source 111 to generate image light. For example, a transmissive or reflective liquid crystal panel or a digital mirror device can be used as the optical modulation element. In this embodiment, the case where the optical modulator 113 includes a transmissive liquid crystal panel 115 as the optical modulation element will be described as an example. The optical modulator 113 includes three liquid crystal panels 115 corresponding to the three primary colors: red, green, and blue. The light modulated by the liquid crystal panels 115 is incident on the optical unit 117 as image light. Hereinafter, red will be denoted as "R", green as "G", and blue as "B".

[0107] An optical modulator 113 is connected to an optical modulator drive circuit 123. The optical modulator drive circuit 123 drives the optical modulator 113 to draw images on the liquid crystal panel 115 in frame units. The optical unit 117 is equipped with optical elements such as lenses and mirrors, and projects the image light modulated by the optical modulator 113 toward the screen SC. An image based on the image light projected by the optical unit 117 is formed on the screen SC. The image formed on the screen SC by the image light projected by the projection unit 110 is called the projected image.

[0108] [Operation / Input System] The projector 100 includes an operation panel 131, a remote control receiver 133, and an input interface 135. The input interface 135 is connected to a bus 180 and communicates data with the control unit 150 and other components via the bus 180. The operation panel 131 is, for example, located on the housing of the projector 100 and is equipped with various switches. When a switch on the operation panel 131 is operated, the input interface 135 outputs an operation signal corresponding to the operated switch to the control unit 150.

[0109] The remote control receiver 133 receives infrared signals transmitted by the remote controller (remote control). The remote control receiver 133 outputs an operation signal corresponding to the received infrared signal. The input interface 135 outputs the input operation signal to the control unit 150. This operation signal corresponds to the switch on the remote controller that was operated.

[0110] [Spectroscopic Imaging Unit] The spectral imaging unit 137 captures the projection image displayed on the screen SC by the projection unit 110 and outputs spectral imaging data.

[0111] Figure 10 is a schematic diagram of the spectral imaging unit 137. The spectral imaging unit 137 is an example of the "spectroscopic imaging device" of the present invention. The spectral imaging unit 137 comprises an incident optical system 301 into which ambient light is incident, a spectroscopic element 302 that spectrally analyzes the incident light, and an imaging element 303 that images the light spectrally analyzed by the spectroscopic element 302.

[0112] The incident optical system 301 is configured, for example, as a telecentric optical system, and guides the incident light to the spectroscopic element 302 and the image sensor 303 so that the optical axis and the principal ray are parallel or approximately parallel. The spectroscopic element 302 uses a tunable interference filter that includes a pair of opposing reflective films 304 and 305, and a gap changing unit 306 that can change the gap dimension of these reflective films 304 and 305. The gap changing unit 306 is configured, for example, as an electrostatic actuator. A tunable interference filter is also called an etalon. This spectroscopic element 302 is positioned on the optical path of the light incident on the image sensor 303.

[0113] The spectroscopic element 302 can change the gap dimensions of the reflective films 304 and 305 by changing the voltage applied to the gap changing unit 306 under the control of the control unit 150, thereby changing the spectral wavelength λi (i=1,2,··,N), which is the wavelength of light transmitted through the reflective films 304 and 305. The image sensor 303 is a device that images the light transmitted through the spectroscopic element 302, and is composed of, for example, a CCD or CMOS. The spectroscopic imaging unit 137 sequentially switches the wavelength of the light spectrally analyzed by the spectroscopic element 302 according to the control of the control unit 150, and images the light transmitted through the spectroscopic element 302 with the image sensor 303 and outputs spectral imaging data. The spectral imaging data is data output for each pixel constituting the image sensor 303, and is data indicating the intensity of the light received by that pixel, i.e., the amount of light. The spectral imaging data output by the spectroscopic imaging unit 137 is input to the control unit 150. Since this spectral imaging unit 137 is of the wavelength scanning type, it is easier to obtain spectral imaging data with higher resolution compared to the case of the wavelength dispersive type.

[0114] [g section] As shown in Figure 9, the projector 100 includes a communication unit 139. The communication unit 139 is connected to a bus 180. As shown in Figure 14, which will be described later, the communication unit 139 functions as an interface for mutual data communication between projectors 100 when multiple projectors 100 are connected. In this embodiment, the communication unit 139 is a wired interface for connecting cables, but it may also be a wireless communication interface that performs wireless communication such as wireless LAN or Bluetooth. "Bluetooth" is a registered trademark.

[0115] [Image Processing System] Next, we will explain the image processing system of Projector 100. As shown in Figure 9, the projector 100 includes an image interface 141, an image processing unit 143, and a frame memory 145 as its image processing system. The image processing unit 143 is connected to a bus 180 and communicates data with the control unit 150 and other units via the bus 180.

[0116] The image interface 141 is an interface for receiving image signals and includes a connector to which the cable 3 is connected, and an interface circuit for receiving image signals via the cable 3. The image interface 141 extracts image data and synchronization signals from the received image signals and outputs the extracted image data and synchronization signals to the image processing unit 143. The image interface 141 also outputs a synchronization signal to the control unit 150. The control unit 150 controls other components of the projector 100 in synchronization with the synchronization signal. The image processing unit 143 performs image processing on the image data in synchronization with the synchronization signal.

[0117] An image supply device 200 is connected to the image interface 141 via cable 3. The image supply device 200 can be, for example, a notebook PC (Personal Computer), a desktop PC, a tablet terminal, a smartphone, or a PDA (Personal Digital Assistant). Alternatively, the image supply device 200 may be a video playback device, a DVD player, a Blu-ray disc player, etc. The image signal input to the image interface 141 may be a moving image or a still image, and the data format is arbitrary. Note that the connection is not limited to a wired connection using cable 3, but may also be a wireless connection using wireless communication.

[0118] The image processing unit 143 and the frame memory 145 are composed of, for example, integrated circuits. Integrated circuits include LSIs (Large-Scale Integrated Circuits), ASICs (Application Specific Integrated Circuits), PLDs (Programmable Logic Devices), FPGAs (Field-Programmable Gate Arrays), SoCs (System-on-a-chip), etc. Furthermore, analog circuits may be included as part of the configuration of the integrated circuit.

[0119] The image processing unit 143 is connected to the frame memory 145. The image processing unit 143 expands the image data input from the image interface 141 into the frame memory 145 and performs image processing on the expanded image data.

[0120] The image processing unit 143 performs various processes, including, for example, geometric correction processing to correct trapezoidal distortion of the projected image, and OSD (On Screen Display) processing to superimpose an OSD menu. The image processing unit 143 also performs image adjustment processing to adjust the brightness and hue of the image data, resolution conversion processing to adjust the aspect ratio and resolution of the image data to match the optical modulator 113, and frame rate conversion.

[0121] The image processing unit 143 outputs the processed image data to the optical modulator drive circuit 123. The optical modulator drive circuit 123 generates drive signals for each of the red, green, and blue colors based on the image data input from the image processing unit 143. Based on the generated drive signals for each color, the optical modulator drive circuit 123 drives the corresponding colored liquid crystal panels 115 of the optical modulator 113 and draws an image on each colored liquid crystal panel 115. As light emitted from the light source 111 passes through the liquid crystal panels 115, image light corresponding to the image data is generated.

[0122] [Control Unit / Storage Unit] The control unit 150 includes a storage unit 160 and a processor 170. The storage unit 160 is composed of, for example, a non-volatile semiconductor memory such as flash memory or EEPROM, or an SSD (Solid State Drive) using flash memory. In this embodiment, the case in which the control unit 150 includes the storage unit 160 is described, but the storage unit 160, which is composed of, for example, a hard disk drive, may be provided outside the control unit 150. The storage unit 160 stores a control program 161, image data such as adjustment image data 162 and pattern image data 163, setting data 164, parameters 165, correction parameters 166, and calibration data 167. The control unit 150 and the spectroscopic imaging unit 137 correspond to an example of the "spectroscopic imaging device" of the present invention.

[0123] The control program 161 is a program such as an OS (Operating System) or application program executed by the processor 170. The processor 170 controls and performs calculations on each part according to this control program 161, measures information about the color of the image projected onto the screen SC, and uses the measurement results to correct the projected image. In the following description, this process will be appropriately referred to as "measurement and image quality adjustment process".

[0124] There are two operating modes that define the operating state when performing measurement and image quality adjustment processing: a high-precision mode and a high-speed mode. As shown in Figure 11, the high-precision mode is an operating mode that enables high-precision measurement by setting the number of wavelengths of light dispersed by the spectroscopic element 302 (spectral wavelength λi) to a relatively large number, N1. In contrast, the high-speed mode is an operating mode that shortens the time required for measurement by setting the number of wavelengths of light dispersed by the spectroscopic element 302 (spectral wavelength λi) to a smaller number, N2, than N1.

[0125] Figure 11 shows an example of the high-precision mode, where the spectral wavelength λi is set in 10nm increments within the range of 400nm to 700nm, and N1 is 31. Also in Figure 11, an example of the high-speed mode is shown, where the spectral wavelength λi is set in 40nm increments within the range of 400nm to 680nm, and N2 is 8. Both operating modes are set to an exposure time that allows the image sensor 303 to obtain a sufficient signal-to-noise ratio, and high-precision spectral imaging data can be obtained. For example, when the exposure time is set to 60msec, the measurement time in high-precision mode is 2.07 seconds, and the measurement time in high-speed mode is 0.53 seconds. Note that N1 is an integer of 2 or more, and the values ​​of N1 and N2 can be changed as appropriate, as long as they are within the range of an integer smaller than N1. The high-precision mode is an example of the "first mode" of the present invention, and the high-speed mode is an example of the "second mode" of the present invention.

[0126] Returning to Figure 9, the setting data 164 is data that sets the processing conditions for various processes executed by the processor 170, and for example, it is data that includes the imaging conditions for each operating mode, including N1 and N2 mentioned above. The parameter 165 is, for example, the parameter for the image processing to be executed by the image processing unit 143.

[0127] The image data stored in the memory unit 160 is the data that forms the basis of the image that the projector 100 displays on the screen SC, and includes, for example, pattern image data 163 and adjustment image data 162. The pattern image data 163 is, for example, image data in which marks of a predetermined shape are placed at the four corners. The processor 170 acquires imaging data (which may also be spectral imaging data) when the image corresponding to the pattern image data 163 is projected onto the screen SC. The processor 170 also acquires information (in this configuration, a projection transformation matrix 167b) that associates the pixels projected onto the screen SC with the pixels of the liquid crystal panel 115, based on the acquired imaging data.

[0128] The adjustment image data 162 is, for example, single-color image data for each of the RGB colors. The control unit 150 acquires spectral imaging data for each spectral wavelength λi when the image corresponding to the adjustment image data 162 is projected onto the screen SC, and acquires correction data 167a that corrects each spectral imaging data based on the spectral imaging data. In this case, N1 spectral imaging data are acquired in high-precision mode, and N2 spectral imaging data are acquired in high-speed mode, which is fewer than N1. Based on these acquired spectral imaging data, predetermined measurement targets are measured. The measurement targets are the absolute values ​​of each of the RGB colors and the color unevenness within the projection surface.

[0129] The correction parameter 166 is a parameter generated by the "measurement and image quality adjustment process," and is an image processing parameter used by the image processing unit 143 to correct the absolute value of each color and color unevenness in the input image data. Calibration data 167 includes correction data 167a, projection transformation matrix 167b, estimation matrix M, and transformation data 167c. Correction data 167a is data that corrects the sensitivity distribution of the image sensor 303 and corrects the spectral imaging data so that the output of the image sensor 303 becomes uniform.

[0130] Due to the influence of lens aberrations and other factors in the lens included in the incident optical system 301, the output of each pixel constituting the image sensor 303 is not uniform and varies depending on the pixel's position. In other words, a sensitivity distribution occurs in the image sensor 303. This sensitivity distribution shows that the output decreases in the peripheral parts of the image sensor 303 compared to the center. Therefore, when correcting the color of an image based on spectral imaging data captured by the spectral imaging unit 137, accurate correction may not be possible due to the influence of errors. Furthermore, the sensitivity distribution of the spectral imaging unit 137 is affected by the optical filters that cut ultraviolet and infrared rays coated on the lens surface. That is, the output of the spectral imaging unit 137 also differs depending on the color of the image captured by the spectral imaging unit 137.

[0131] Correction data 167a is generated during the manufacturing of the projector 100, and is also generated for each pixel of the image sensor 303. In addition, multiple correction data 167a are generated corresponding to each of the RGB colors of light projected by the projection unit 110 and the spectral wavelength λi set in the spectral imaging unit 137. By generating correction data 167a for each color and for each spectral wavelength λi, the correction accuracy of the spectral sensitivity of the spectral imaging unit 137 is improved.

[0132] The projection transformation matrix 167b is a transformation matrix that converts the coordinates set on the liquid crystal panel 115 of the optical modulator 113 to the coordinates set on the spectral imaging data. The liquid crystal panel 115 has a configuration in which multiple pixels are arranged in a matrix. The coordinates set on the liquid crystal panel 115 are the coordinates used to identify each pixel arranged in this matrix.

[0133] The estimation matrix M is a matrix used for estimating the spectrum. The estimation matrix M is generated during the manufacturing of the projector 100 and stored in the storage unit 160 as part of the calibration data 167. The estimation matrix M is generated based on the spectral imaging data captured by the spectral imaging unit 137. Optical components are mounted on the optical unit 117, and optical components are also used for the components that guide the light emitted from the light source 111 to the liquid crystal panel 115 of the optical modulator 113. Furthermore, each pixel constituting the liquid crystal panel 115 has spectral characteristics, and errors occur in the wavelength of the image light transmitted by each pixel. Due to the optical characteristics of these optical components, errors occur in the values ​​of the spectral imaging data generated by the spectral imaging unit 137, and the colorimetric accuracy decreases. The calculation methods for the correction data 167a, projection transformation matrix 167b, and estimation matrix M described above can be broadly applied using known methods.

[0134] The conversion data 167c in the memory unit 160 is data that associates the color measurement results obtained from N1 spectral imaging data in high-precision mode with the color measurement results obtained from N2 spectral imaging data in high-speed mode. Because the N2 spectral imaging data has fewer data points than the N1 spectral imaging data, or because the interval between spectral wavelengths λi is wider, the error becomes larger when estimating the spectrum of each RGB color, for example, the error in the peak wavelength of each color becomes larger. When the error in the peak wavelength of each color becomes larger, the measurement accuracy of the absolute value of each color decreases. In this embodiment, the conversion data 167c is measured in advance during the manufacturing of the projector 100, and by using the conversion data 167c when estimating the spectrum from the N2 spectral imaging data, the deviation of the peak wavelength of each color is kept within an acceptable range. This improves the measurement accuracy and correction accuracy of the absolute value of each color.

[0135] [processor] The processor 170 is an arithmetic processing unit composed of, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), a microcontroller, etc. The processor 170 may be composed of a single processor or a combination of multiple processors. The processor 170 functions as a projection control unit 171, an imaging control unit 173, and an arithmetic unit 175, etc., by executing a control program 161 stored in the memory unit 160.

[0136] The projection control unit 171 controls the image displayed on the screen SC by the projection unit 110. Specifically, the projection control unit 171 controls the image processing unit 143 to perform image processing on the image data input from the image interface 141. In this case, the projection control unit 171 may read the parameters 165 necessary for image processing and OSD menu information from the storage unit 160 and output them to the image processing unit 143. The projection control unit 171 can also adjust the brightness of the light source 111 by controlling the light source drive circuit 121.

[0137] The imaging control unit 173 instructs the spectral imaging unit 137 to perform imaging. In this case, the imaging control unit 173 sets imaging conditions for high-precision mode or high-speed mode based on the setting data 164 in the storage unit 160. The calculation unit 175 performs calculations to measure the absolute values ​​of each RGB color and the color unevenness within the projection surface based on the multiple spectral imaging data output from the spectral imaging unit 137. Furthermore, the calculation unit 175 generates correction parameters 166 to correct the absolute values ​​of each color and the color unevenness based on the measurement results obtained from the calculations. The image processing unit 143 can generate corrected image data with corrected absolute values ​​of each color and color unevenness by using the correction parameters 166 when processing the image data input from the image interface 141.

[0138] [Projector operation] Figure 12 is a flowchart showing the operation of the display system 1 related to measurement and image quality adjustment processing. When the control unit 150 receives an operation signal corresponding to the OSD menu display instruction via the input interface 135, the projection control unit 171 performs a process to superimpose the OSD menu onto the projected image, thereby displaying the OSD menu (step S101). This OSD menu includes keys to instruct image quality adjustment for high-precision mode and keys to instruct image quality adjustment for high-speed mode, allowing the user to select between high-precision mode and high-speed mode via the OSD menu. The OSD menu and operation method for selecting high-precision mode and high-speed mode may be modified as appropriate. For example, the modes may be selectable using the operation panel 131 or a remote control.

[0139] Image quality adjustment in this flowchart involves correcting the image using the imaging results from the spectral imaging unit 137. More specifically, it involves correcting the absolute values ​​of each RGB color and color unevenness in the projected image to predetermined conditions. These predetermined conditions are, for example, conditions corresponding to the image quality at the time of manufacture. If image quality adjustment for high-precision mode is instructed (step S102 / high-precision mode), the control unit 150 acquires the imaging conditions for high-precision mode included in the setting data 164 (step S103). Next, the control unit 150 performs imaging processing to capture a projection image according to the acquired imaging conditions (step S104) and generates N1 spectral imaging data (step S105).

[0140] In the imaging process, first, the control unit 150 displays a pattern image on the screen SC based on the pattern image data 163 in the storage unit 160, and controls the spectral imaging unit 137 to capture the pattern image. The data from the captured pattern image is, for example, data captured by the spectral imaging unit 137 with the wavelength fixed to a predetermined wavelength. Next, the control unit 150 calculates a projection transformation matrix 167b showing the correspondence between imaging coordinates and panel coordinates based on the data from the captured pattern image, and stores it in the storage unit 160. Then, the control unit 150 displays an adjustment image on the screen SC based on the adjustment image data 162 in the storage unit 160, and generates N1 spectral imaging data by changing the spectral wavelength λi of the spectral imaging unit 137 according to the acquired imaging conditions. The N1 spectral imaging data is an example of the "N1 first imaging data" of the present invention.

[0141] Next, the control unit 150, using the calculation unit 175, measures the absolute values ​​of each RGB color and the color unevenness within the projection surface based on N1 spectral imaging data (step S106). For example, the spectrum of each color is estimated from the N1 spectral imaging data, and the absolute value of each color is obtained from each spectrum. In addition, the color unevenness within the projection surface is obtained by detecting the color unevenness of each pixel (differences in brightness, differences in the estimated spectrum, etc.) from the spectral imaging data. Note that known methods can be widely applied to the measurement methods for the absolute values ​​of each color and the color unevenness.

[0142] The control unit 150 generates correction parameters 166 for correcting the absolute value of each measured color and color unevenness (step S107). The generated correction parameters 166 are stored in the storage unit 160. These correction parameters 166 may be correction parameters that use one pixel constituting the liquid crystal panel 115 as a unit, or correction parameters that use multiple pixels as a unit. When generating correction parameters 166 that use multiple pixels as a unit, the correction parameters 166 for pixels for which correction parameters 166 have not been generated can be obtained, for example, by interpolation calculation using linear interpolation.

[0143] When the image supply device 200 starts supplying an image signal, the control unit 150 reads the correction parameter 166 from the storage unit 160 and outputs it to the image processing unit 143. When the image interface 141 starts receiving an image signal and image data is input from the image interface 141, the image processing unit 143 expands the input image data into the frame memory 145. The image processing unit 143 corrects the image data using the correction parameter 166 and other inputs from the control unit 150 and displays the image corresponding to the image data on the screen SC (step S108). The supplied image signal is an example of the "first image data" of the present invention, and the corrected image data is an example of the "first corrected image data" of the present invention. Furthermore, the projected image corresponding to the corrected image data is an example of the "first projected image" of the present invention.

[0144] If image quality adjustment for high-speed mode is instructed in step S102 (step S102 / high-speed mode), the control unit 150 acquires the imaging conditions for high-speed mode included in the setting data 164 (step S109). Next, the control unit 150 performs imaging processing to capture a projection image according to the acquired imaging conditions (step S110) and generates N2 spectral imaging data (step S111). This imaging processing is the same as the imaging processing in step S104, except that the control of the spectral wavelength λi of the spectrometer 302 is different. In high-speed mode, the number of spectral wavelengths λi is smaller than in high-precision mode, so the measurement can be completed in a shorter time. The N2 spectral imaging data is an example of the "N2 second imaging data" of the present invention.

[0145] Next, the control unit 150 uses the calculation unit 175 to estimate the spectrum from N2 spectral imaging data (step S112). In this case, the calculation unit 175 converts the spectra of each color obtained from the N2 spectral imaging data into equivalent spectra obtained from N1 spectral imaging data based on the conversion data 167c in the storage unit 160. As a result, a spectrum is estimated in which the peak wavelengths of each color are located at approximately the same wavelengths as the spectrum obtained from N1 spectral imaging data.

[0146] The control unit 150 measures the absolute value of each color and the color unevenness within the projection surface based on N2 spectral imaging data (step S113). For example, the absolute value of each color is obtained from the spectrum estimated in step S112. Color unevenness within the projection surface is obtained by detecting pixel-by-pixel color unevenness (differences in brightness, etc.) from the spectral imaging data. A wide range of known methods can be applied to measure the absolute value of each color and the color unevenness.

[0147] The control unit 150 measures the absolute value of each color and the color unevenness within the projection surface based on N2 spectral imaging data, and then executes the processes in steps S107 and S108. As a result, the control unit 150 generates correction parameters 166 to correct the measured absolute values ​​of each color and the color unevenness, corrects the image data using the correction parameters 166, and displays the image corresponding to the corrected image data on the screen SC. The corrected image data is an example of the "second corrected image data" of the present invention, and the projection image corresponding to the corrected image data is an example of the "second projection image" of the present invention.

[0148] As described above, in the second embodiment, when the display system 1 is in high-precision mode, the spectral imaging unit 137, which is equipped with an image sensor 303 and a spectral element 302, generates N1 spectral imaging data by capturing the projected image with different spectral wavelengths of the spectral element 302. The projector 100 generates corrected image data by correcting the input image data based on the N1 spectral imaging data. Then, the projector 100 generates a projected image based on the corrected image data and projects it onto the screen SC, which is the projection surface.

[0149] On the other hand, when the display system 1 is in high-speed mode, the spectral imaging unit 137 generates N2 spectral imaging data by capturing the projected image with different spectral wavelengths of the spectral element 302. The projector 100 generates corrected image data by correcting the input image data based on the N2 spectral imaging data. Then, the projector 100 generates a projected image based on the corrected image data and projects it onto the screen SC.

[0150] As mentioned above, N1 is an integer greater than or equal to 2, and N2 is an integer less than N1. Therefore, the high-precision mode can be used when precise image quality adjustment is desired or in cases where image quality is important, such as for theater use. On the other hand, when a simple check is desired, the high-speed mode allows for checking the image quality in a shorter time compared to the high-precision mode. Thus, user convenience is improved when adjusting or checking image quality. Examples of situations where a simple check is desired include checking for projection abnormalities, especially when a check needs to be performed in the short time between projections, or when checking in cases where image quality is not a high priority, such as in offices or educational settings.

[0151] Furthermore, in high-precision mode, predetermined color information is measured based on N1 first imaging data (spectroscopic imaging data), and the input image data is corrected based on the measurement results to obtain the corrected image data. In this embodiment, the predetermined color information was the absolute values ​​of each RGB color and the color unevenness within the projection surface, but either one of them may be used, or it may be information for detecting stains or deposits, and any appropriate information can be applied. This makes it possible to adjust the image quality in terms of color using the spectroscopic element 302.

[0152] Furthermore, conversion data 167c is acquired in advance to correlate the color spectral results obtained from N1 spectral imaging data in high-precision mode with the color spectral results obtained from N2 spectral imaging data in high-speed mode. In high-speed mode, the color spectral results obtained from N2 spectral imaging data are converted into information equivalent to the color spectral results obtained from N1 spectral imaging data based on conversion data 167c. Then, the input image data is corrected based on the converted information, and this corrected image data is used. This makes the shifts in the peak wavelengths of each color that occur when estimating the spectrum with a relatively small number of spectral imaging data points to an acceptable range, thereby maintaining sufficient measurement accuracy and enabling appropriate correction.

[0153] Furthermore, the projector 100 projects an image that includes an OSD menu with options for high-precision mode and high-speed mode, making it easy to select each mode. This reduces the number of switches compared to when switches for selecting each mode are provided on the control panel 131 or remote control, and also makes it easier to reuse existing control panels and remote controls.

[0154] Furthermore, the spectroscopic element 302 has a pair of reflective films 304 and 305 and a gap changing section 306 that can change the gap dimension between the pair of reflective films 304 and 305, and is a tunable interference filter positioned on the optical path of light incident on the image sensor 303. This makes it easier to perform measurements ranging from high-precision to short-time measurements while maintaining high resolution, compared to the case of a wavelength-dispersive type. Although the example given shows two operating modes consisting of a high-precision mode and a high-speed mode, it is also possible to add operating modes with different imaging conditions such as the number of spectral wavelengths λi and exposure time, resulting in three or more operating modes.

[0155] For example, in the example shown in Figure 11, the S / N ratio of the measured values ​​is increased and measurement reproducibility is improved by making the exposure time the same as in the high-speed mode, but this is not the only example. Other high-speed modes may be provided that shorten the measurement time by making the exposure time shorter than in the high-speed mode. If the high-speed mode shown in Figure 11 is tentatively labeled as the second mode, other high-speed modes may be labeled as the third mode, etc. Alternatively, the high-speed mode shown in Figure 11 may be omitted, and other high-speed modes may be presented as examples of the "second mode" of the present invention.

[0156] In the second embodiment, an example was given in which a configuration equivalent to a spectroscopic imaging device is integrated with the projector 100. However, as shown in Figure 13, the display system 1A may also include a spectroscopic imaging device 400 separate from the projector 100. Furthermore, although Figure 13 shows a wired connection between the projector 100 and the spectroscopic imaging device 400 using a cable 4, a wireless connection using wireless communication may also be used.

[0157] [Third Embodiment] Figure 14 shows a display system 1 according to the third embodiment. In the third embodiment, two projectors 100 are connected by a cable 5, and data communication is performed between the projectors 100 to synchronize the colors of the images displayed on the screen SC by each projector 100. When viewed from the direction opposite the screen SC, the projector 100 that displays an image on the left side of the screen SC is referred to as projector 100A, and the projector 100 that displays an image on the right side of the screen SC is referred to as projector 100B. The number of projectors 100 connected is not limited to two, but may be three or more. In addition, a wireless connection using wireless communication may be used instead of a wired connection using cable 3.

[0158] The configurations of projector 100A and projector 100B are the same as those of projector 100 shown in Figure 9, so their illustrations are omitted. For the sake of explanation, the components of projector 100A will be denoted by the symbol "A" in Figure 9, and the components of projector 100B will be denoted by the symbol "B" in Figure 9. For example, the control unit 150 of projector 100A will be referred to as "control unit 150A," and the control unit 150 of projector 100B will be referred to as "control unit 150B." Projector 100A and projector 100B are each connected to the image supply device 200 via cable 3, and display images on the screen SC based on image signals supplied from the image supply device 200.

[0159] Projector 100A operates as the master unit, and Projector 100B operates as the slave unit. That is, Projector 100B operates according to the control of Projector 100A. As the master unit, Projector 100A instructs Projector 100B to calculate the values ​​necessary for generating the correction parameter 166, and instructs Projector 100B to perform image processing using the correction parameter 166.

[0160] The area onto which projector 100A projects image light is denoted as "projection area 20A," and the area onto which projector 100B projects image light is denoted as "projection area 20B." Projection area 20A and projection area 20B partially overlap.

[0161] In the third embodiment, projector 100A has a function to accept color matching between projectors 100A and 100B that are connected to each other. For example, the control unit 150A displays an OSD menu including a key for instructing color matching via the projection control unit 171A, and accepts color matching instructions from the user. In this case, the control unit 150A displays a key for instructing color matching in high-precision mode and a key for instructing color matching in high-speed mode. The key for instructing color matching in high-precision mode is an example of a "key for instructing image quality adjustment in high-precision mode," and the key for instructing color matching in high-speed mode is an example of a "key for instructing image quality adjustment in high-speed mode."

[0162] When high-precision color matching mode is instructed, the control unit 150A causes the control units 150A and 150B to perform the processing steps S103 to S107 shown in Figure 12. This generates the correction parameter 166A for projector 100A and the correction parameter 166B for projector 100B. This generates the correction parameters 166A and 166B corresponding to the correction parameter 166 in the high-precision mode of the second embodiment. In this case, projector 100B may perform only the imaging processing corresponding to steps S103 to S105 in Figure 12 using the spectral imaging unit 137B, and transmit N1 spectral imaging data obtained by imaging to projector 100A. In this case, projector 100A generates the correction parameter 166B for projector 100B by performing the remaining steps S106 to S107 using the spectral imaging data transmitted from projector 100B.

[0163] The control unit 150A corrects the image data input to each projector 100A and 100B using correction parameters 166A and 166B, and displays the corresponding image on the screen SC. This ensures that the projected images from each projector 100A and 100B are color-matched. While these correction parameters 166A and 166B correct the absolute values ​​and color unevenness of each color in each projector 100A and 100B, other correction methods capable of matching the colors of projectors 100A and 100B may also be used.

[0164] For example, the control unit 150A acquires N1 spectral imaging data obtained by the spectral imaging unit 137A of projector 100A and N1 spectral imaging data obtained by the spectral imaging unit 137B of projector 100B. Next, the control unit 150A measures the color difference between projectors 100A and 100B based on the acquired spectral imaging data. Subsequently, the control unit 150A may generate correction parameters to correct the color displayed by projector 100B to the color of projector 100A based on the measurement results.

[0165] When high-speed color matching is instructed, the control unit 150A sequentially causes the control units 150A and 150B to execute the processes of steps S109 to S113, S107, and S108 shown in Figure 12. This generates the correction parameter 166A for projector 100A and the correction parameter 166B for projector 100B. This generates the parameters 166A and 166B corresponding to the correction parameter 166 in the high-speed mode of the second embodiment. In this case, projector 100B may perform only the imaging-related processing corresponding to steps S109 to S111 in Figure 12 using the spectral imaging unit 137B, and transmit N2 spectral imaging data obtained by imaging to projector 100A. In this case, projector 100A generates the correction parameter 166B for projector 100B by performing the remaining steps S112 to S113 and S107 using the spectral imaging data transmitted from projector 100B.

[0166] Even in high-speed mode, the control unit 150A may measure the color difference between projectors 100A and 100B and generate correction parameters based on the measurement results to correct the color displayed by projector 100B to the color of projector 100A. In this case, compared to the case where a high-speed mode is adopted in which correction parameters 166A and 166B are generated to correct the absolute values ​​of each color of projectors 100A and 100B and color unevenness, the time required to generate the correction parameters can be reduced, and the high-speed mode can be made even shorter.

[0167] In other words, in the display system 1 of the third embodiment, when in high-precision mode, for each projected image from each projector 100A and 100B, the spectral imaging units 137A and 137B generate N1 spectral imaging data captured at different spectral wavelengths. Next, either projector 100A or 100B generates corrected image data by correcting the input image data based on the N1 spectral imaging data. The N1 spectral imaging data are an example of the "N1 first imaging data" of the present invention, and the input image data is an example of the "first image data" of the present invention. Furthermore, the corrected image data is an example of the "first corrected image data" of the present invention. Next, projectors 100A and 100B project images based on the corrected image data onto the screen SC, which is the projection surface. The projected image is an example of the "first projected image" of the present invention.

[0168] On the other hand, when the display system 1 is in high-speed mode, for each projected image from each projector 100A and 100B, the spectral imaging units 137A and 137B generate N2 spectral imaging data captured at different spectral wavelengths. Next, either projector 100A or 100B generates corrected image data by correcting the input image data based on the N2 spectral imaging data. Then, each projector 100A and 100B projects the image based on the corrected image data onto the screen SC, which is the projection surface. Note that the N2 spectral imaging data are an example of the "N2 second imaging data" of the present invention, the corrected image data is an example of the "second corrected image data" of the present invention, and the projected image is an example of the "second projected image" of the present invention.

[0169] In this way, the high-precision mode can be used when precise color matching of each projector 100A and 100B is required, or in cases where image quality is paramount, such as for theater applications. On the other hand, when a simple check is desired, the high-speed mode allows for checking image quality in a shorter time compared to the high-precision mode. Therefore, user convenience is improved when adjusting and checking image quality when using multiple projectors 100. It should be noted that the configuration equivalent to the spectral imaging device is not limited to being provided in each of the projectors 100A and 100B; for example, the configuration equivalent to the spectral imaging device may be provided in only one of the projectors 100A or 100B. Also, as shown in Figure 15, the display system 1B may include a spectral imaging device 400 separate from the projectors 100A and 100B, and this spectral imaging device 400 may capture the projected images of the projectors 100A and 100B.

[0170] The present invention is not limited to the configurations of the embodiments described above, and can be implemented in various forms without departing from its essence. In the embodiments described above, the present invention was applied to the display systems 1 and 1A, projectors 100, 100A, and 100B, and their control methods, as shown in Figure 9, etc., but the invention is not limited thereto. For example, the example given was the correction of at least one of the absolute values ​​of each RGB color, color unevenness within the projection surface, and color between multiple projectors 100, but the measurement target and correction target may be changed as appropriate. Also, the example given was that the measurement target light is visible light, but it may be other types of light, such as infrared or far-infrared light. Furthermore, the example given was that a tunable interference filter is used for the spectroscopic element 302, but other wavelength scanning filters may be used.

[0171] Furthermore, either transmissive or reflective liquid crystal panels may be applied to the three liquid crystal panels 115 of the optical modulator 113. Alternatively, a configuration combining one liquid crystal panel and a color wheel may be applied instead of the three liquid crystal panels 115. In addition, various known methods such as a method using three digital mirror devices (DMDs) or a DMD method combining one digital mirror device and a color wheel may be applied to the optical modulator 113.

[0172] Furthermore, the configuration of each part shown in Figure 9, etc., may be implemented in hardware, or it may be implemented through the collaboration of hardware and software, and is not limited to a configuration in which independent hardware resources are arranged as shown in the figure.

[0173] Furthermore, the processing units in the flowchart shown in Figure 12 represent divisions of the processing performed by the control unit 150 according to the main processing content. The method of division and the names of the processing units in each flowchart do not limit the embodiments. Also, the processing order in the flowchart described above is not limited to the example shown.

[0174] Furthermore, the control program 161 may be stored in an external device or apparatus and retrieved via the communication unit 139 or the like. It can also be recorded on a recording medium that is readable by a computer. As the recording medium, magnetic, optical, or semiconductor memory devices can be used. Specifically, examples include portable or fixed recording media such as flexible disks, various optical disks, magneto-optical disks, flash memory, and card-type recording media. The recording medium may also be a non-volatile storage device such as RAM, ROM, or HDD, which are internal storage devices of the image display device. [Explanation of symbols]

[0175] 1…Display system, 3…Cable, 4…Cable, 5…Cable, 10…Imaging device, 11…Incident optical system, 12…Spectroscopic element, 13…Image sensor, 14…Light source unit, 14a…Substrate, 15…Reflective film, 16…Reflective film, 17…Gap changing unit, 20A…Projection area, 20B…Projection area, 30…Display device, 40…Storage device, 50…Processing device, 60…Control unit, 61…Imaging control unit, 70…Processing unit, 71…Measurement bandwidth indicator unit, 80…Storage unit, 81…Measurement bandwidth data storage unit, 82…Setting data storage unit, 83…Conversion Data storage unit, 100...Projector, 100A...Projector, 100B...Projector, 101...First substrate, 102...Second substrate, 103...Third substrate, 104...Reflective film, 105...Electrostatic actuator, 106...Bonding layer, 110...Projection unit, 111...Light source, 112...Spectroscopic element, 113...Optical modulator, 115...Liquid crystal panel, 117...Optical unit, 120...Drive unit, 121...Light source drive circuit, 123...Optical modulator drive circuit, 131...Operation panel, 133...Remote control receiver, 135...Input Interface, 137...Spectroscopic imaging unit, 137A...Spectroscopic imaging unit, 137B...Spectroscopic imaging unit, 139...Communication unit, 141...Image interface, 143...Image processing unit, 145...Frame memory, 150...Control unit, 150A...Control unit, 150B...Control unit, 160...Storage unit, 161...Control program, 162...Adjustment image data, 163...Pattern image data, 164...Setting data, 165...Parameters, 166...Correction parameters, 166A...Correction parameters, 166B...Correction parameters ,167...Calibration data, 167a...Correction data, 167b...Projection transformation matrix, 167c...Transformation data, 170...Processor, 171...Projection control unit, 171A...Projection control unit, 173...Imaging control unit, 175...Calculation unit, 180...Bus, 200...Image supply device, 301...Incident optical system, 302...Spectrophotometer, 303...Image sensor, 304...Reflective film, 305...Reflective film, 306...Gap changing unit, 400...Spectrophotometer, 400A...Spectrophotometer, M1...High-precision mode, M2...High-speed mode, M3...Optimal mode.

Claims

1. A control method for a spectroscopic imaging apparatus comprising an image sensor and a spectroscopic element, When the spectroscopic imaging device is in the first mode, The spectroscopic imaging device generates a first measurement spectrum consisting of N1 wavelengths, which are two or more integers, obtained by imaging the object with different output wavelengths of the spectroscopic elements. When the spectroscopic imaging device is in the second mode, The spectroscopic imaging device generates a second measurement spectrum consisting of N2 wavelengths, which are integers smaller than N1, obtained by varying the output wavelengths of the spectroscopic elements to image the object. The spectrum of the object is estimated by converting the second measurement spectrum obtained in the second mode using conversion data that estimates a spectrum consisting of N1 wavelengths from a spectrum consisting of N2 wavelengths. When the spectroscopic imaging device is in the third mode, Determine the number of measurements taken by the spectroscopic imaging device, A control method for a spectroscopic imaging device, in which, when the number of measurements is two or more, a wavelength band that can be thinned out from the shape of a spectrum consisting of N1 wavelengths is selected based on at least one of the spectra of the object estimated by converting the first measurement spectrum generated in the first mode and the second measurement spectrum generated in the second mode, and a spectrum consisting of fewer than N1 wavelengths and with different intervals between the output wavelengths is generated.

2. A control method for a spectroscopic imaging apparatus according to claim 1, A control method for a spectroscopic imaging device, wherein the spectroscopic element is a tunable interference filter positioned on the optical path of light incident on the image sensor, and has a pair of reflective films and a gap changing section that can change the gap dimension of the pair of reflective films.

3. A spectroscopic imaging apparatus comprising an image sensor and a spectroscopic element, A first mode generates a first measurement spectrum consisting of N1 wavelengths, which are integers of two or more, obtained by imaging an object with different output wavelengths of the spectroscopic element. By varying the output wavelength of the spectroscopic element, N2 second measurement spectra are generated, which are integers smaller than N1 obtained by imaging the object. A second mode in which the spectrum of the object is estimated by converting the second measurement spectrum obtained in the second mode using conversion data that estimates a spectrum consisting of N1 wavelengths from a spectrum consisting of N2 wavelengths, Determine the number of measurements taken by the spectroscopic imaging device, A spectroscopic imaging device having a third mode in which, when the number of measurements is two or more, a wavelength band that can be thinned out from the shape of a spectrum consisting of N1 wavelengths is selected based on at least one of the spectra of the object estimated by converting the first measurement spectrum generated in the first mode and the second measurement spectrum generated in the second mode, and a spectrum consisting of fewer wavelengths than N1 and with different intervals between the output wavelengths is generated.

4. A spectroscopic imaging apparatus according to claim 3, The spectroscopic imaging device is a tunable interference filter positioned on the optical path of light incident on the image sensor, having a pair of reflective films and a gap changing section that can change the gap dimension of the pair of reflective films.

5. A computer program for identifying an object based on imaging data from a spectroscopic imaging device equipped with an image sensor and a spectroscopic element, In the first mode of the spectroscopic imaging device, the process involves imaging the target object with different output wavelengths and generating a first measurement spectrum consisting of N1 wavelengths that are integers of 2 or more, In the second mode of the spectroscopic imaging device, the process involves imaging the object with different output wavelengths and generating a second measurement spectrum consisting of N2 wavelengths that are integers smaller than N1, A computer program that causes a computer to execute at least one of the following: In the second mode of the spectroscopic imaging device, the spectrum of the object is estimated by converting the second measurement spectrum obtained in the second mode using conversion data that estimates a spectrum consisting of N1 wavelengths from a spectrum consisting of N2 wavelengths. A computer program that, in the third mode of the spectroscopic imaging device, determines the number of measurements performed by the spectroscopic imaging device, and when the number of measurements is two or more, causes the computer to perform a process to generate a spectrum consisting of fewer than N1 wavelengths, based on at least one of the spectra of the object estimated by converting the first measurement spectrum generated in the first mode and the second measurement spectrum generated in the second mode, by selecting wavelength bands that can be thinned out from the shape of a spectrum consisting of N1 wavelengths, and generating a spectrum consisting of fewer than N1 wavelengths and with different intervals between the output wavelengths.

6. A computer program for identifying an object based on imaging data from a spectroscopic imaging device comprising an image sensor and a spectroscopic element, as described in claim 5, A computer program for identifying an object based on imaging data from a spectroscopic imaging device comprising an image sensor and a spectroscopic element, wherein the spectroscopic element of the spectroscopic imaging device is a tunable interference filter positioned on the optical path of light incident on the image sensor, having a pair of reflective films and a gap changing section that can change the gap dimension of the pair of reflective films.

Images