Combined optical filter multispectral imaging sensor and method

Abstract

The invention discloses a combined optical filter multispectral imaging sensor and method. Relates to the technical field of sensors, in particular to the technical field of combined optical filter multispectral imaging. According to the method, the problem that wide-range and high-precision spectrum adjustment cannot be considered in the prior art is effectively solved, and the precision of multispectral spectrum section selection and weak and small target detection and recognition is improved. The imaging sensor comprises an imaging sensor, an optimized multispectral optical filter array and an optimized liquid crystal tunable optical filter; the imaging method comprises the following steps: a target background image enters the combined optical filter multispectral sensor to obtain narrow-band light of N wavebands and light to be tuned; converting the narrow-band light of the N wavebands and the light to be tuned into image signals; recovering a three-dimensional spectral image, and carrying out rough detection by combining an optical filter multispectral sensor; finely adjusting the combined optical filter multispectral sensor; and spectral imaging.

Description

Technical Field

[0001] The invention relates to the field of sensor technology, and in particular to the field of combined filter multi-spectral imaging technology. Background Art

[0002] Under the current industrial technology background, multispectral imaging sensors are classified according to filter technology, mainly including tunable multispectral imaging sensors, multispectral filter array imaging sensors and filter wheel imaging sensors. Filter wheel imaging sensors are less used due to their large size and mechanical noise. Tunable multispectral imaging sensors mainly include acousto-optic tunable (AOTF) and liquid crystal tunable (LCTF). Both of them achieve the effect of light splitting by changing the refractive index of the crystal to screen the wavelength. However, the acousto-optic tunable filter (AOTF) is more complex to drive, has obvious nonlinear effects, and has a larger system volume than LCTF. In addition, the AOTF has a small aperture and a large optical path loss. In addition, the temperature-controlled tuning speed is slow, and the mechanical tuning has the disadvantages of inconvenient operation and limited band. Liquid crystal tunable multispectral imaging sensors have the advantages of narrow passband width, wide tuning range, low control voltage, large aperture, and high imaging quality, but the working range of single modulation is narrow, and in the visible light band, only a spectral band with a bandwidth of less than 200nm can be modulated at one time. The multi-spectral filter array imaging sensor has a wide operating spectrum, but due to the limitations of the spectral filter array, it has the disadvantages of a small number of wavelengths and insufficient spectral adjustment accuracy.

[0003] Multispectral imaging sensors can be classified into traditional multispectral systems and snapshot multispectral systems according to the information acquisition method. Traditional multispectral imaging systems can be divided into "point scanning", "push scanning (line scanning)" and "staring (wavelength scanning)" according to different scanning methods. Most traditional multispectral imaging systems have complex optical structures, small field of view, and large size, which do not meet the development needs of miniaturization and high quality. On the contrary, compared with scanning multispectral systems, snapshot multispectral imaging does not require scanning and can obtain a complete three-dimensional data cube within a single exposure time period; it can also adapt to fast-moving scenes and solve problems such as optical system detection delays caused by scanning and camera synchronization.

[0004] Therefore, a liquid crystal tunable filter is placed in the center of the multispectral filter array, and a multispectral filter array is placed in front of the detector to achieve the combination of two different multispectral detection methods. The surrounding multispectral filter array provides observation capabilities in a wide spectral range, while the central liquid crystal filter provides precise control of specific wavelengths. Fast and accurate multispectral spectrum selection and detection and identification of weak targets are achieved. At present, there is no multispectral imaging sensor that combines snapshot imaging sensors with tunable functions. Summary of the invention

[0005] In response to the above problems, the present invention discloses a combined filter multi-spectral imaging sensor and method, which relates to the technical field of sensor technology, effectively solves the problem that the prior art cannot take into account both wide range and high precision spectral adjustment, and improves the accuracy of multi-spectral spectrum selection and weak target detection and identification.

[0006] The combined filter multispectral imaging sensor comprises: an imaging sensor, an optimized multispectral filter array and an optimized liquid crystal tunable filter;

[0007] The optimized multi-spectral filter array is installed in front of the imaging sensor target surface, and a circular light-transmitting hole is set in the central area of ​​the optimized multi-spectral filter array, and the optimized liquid crystal tunable filter is set in the circular light-transmitting hole.

[0008] Furthermore, the size of the optimized multispectral filter array is the same as the size of the imaging sensor target surface; the optimization of the multispectral filter array is specifically as follows: Optimize the spacer layer: change the transmitted wavelength by changing the thickness of the spacer layer, where λ represents c length, n represents the refractive index of the filter, d represents the thickness of the spacer layer, k represents the wave number of the incident light, Φ1 and Φ2 represent the phase change of the two surfaces of the filter, and m represents the interference order of the incident light.

[0009] Furthermore, the optimization of the liquid crystal tunable filter is specifically as follows: a Lyot structure is selected as the basic structure of the liquid crystal tunable filter, a two-layer cascaded Lyot structure is used instead of a multi-layer cascade, and the thickness of the control liquid crystal box is set to 0.7 mm.

[0010] Furthermore, the imaging method is implemented by using the combined filter multispectral imaging sensor according to any one of claims 1 to 3, and the method specifically comprises:

[0011] S1, the target background image is injected into the joint filter multi-spectral sensor to obtain N bands of narrow band light and light to be tuned;

[0012] S2, converting the narrow-band lights of N bands and the light to be tuned into image signals;

[0013] S3, restore the three-dimensional spectral image, coarse detection combined with filter multi-spectral sensor;

[0014] S4, fine-tuning combined filter multispectral sensor;

[0015] S5. Spectral imaging.

[0016] Furthermore, the narrow-band lights of the N bands are obtained by spectroscopic analysis by an optimized multi-spectral filter array; and the light to be tuned is obtained by filtering by an optimized liquid crystal tunable filter.

[0017] Furthermore, the conversion of the narrow-band light of N bands and the light to be tuned into the image signal is specifically as follows: the narrow-band light of N bands and the light to be tuned are emitted to the imaging optical path to obtain spectral cube data and spectral image; the imaging optical path is specifically as follows: the imaging sensor target surface converts the received light signal into an electrical signal and transmits it to a computer, and the computer converts the electrical signal into an image signal.

[0018] Furthermore, the three-dimensional spectral image restoration method is specifically: restoring the missing grayscale values ​​of the spectral cube data through an inversion algorithm to obtain a three-dimensional spectral image; the method of coarse detection combined with a filter multi-spectral sensor is specifically: calculating the average grayscale value of N bands in the three-dimensional spectral image, deleting the bands with grayscale values ​​higher than the average grayscale value, and obtaining the target band.

[0019] Furthermore, the method for finely adjusting the combined filter multi-spectral sensor is specifically as follows: according to the target band, the control parameters of the optimized liquid crystal tunable filter are adjusted to make the filter band match the target band, thereby obtaining a finely adjusted combined filter multi-spectral sensor.

[0020] Furthermore, the spectral imaging is specifically as follows: the light emitted by the detection target is emitted into the precisely adjusted combined filter multi-spectral sensor, the light of the target background image is filtered out, and the light to be imaged is obtained, and the light to be imaged is emitted to the imaging light path to obtain spectral imaging.

[0021] The beneficial effects of the present invention are:

[0022] The multi-spectral sensor for joint detection uses the characteristics of a multi-spectral filter array with a wide working spectrum and fast data acquisition. It can quickly acquire multi-band wide spectrum data and obtain several spectrum bands suitable for target detection. The liquid crystal tunable filter has the characteristics of continuous tuning and narrow passband width. By adjusting the central wavelength of the liquid crystal tunable filter, the band can be finely adjusted. In this way, accurate selection of multi-spectral spectrum bands and detection and identification of weak targets can be achieved, while having the advantages of small system size and fast imaging speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 Schematic diagram of a combined filter multi-spectral imaging sensor in an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of optimization of a multi-spectral filter array in an embodiment of the present invention;

[0025] Figure 3Schematic diagram of optimization of a liquid crystal tunable filter in an embodiment of the present invention;

[0026] Figure 4 It is a schematic diagram of the simulation results of the inversion algorithm in an embodiment of the present invention;

[0027] Figure 5 Schematic diagram of the result of rough detection in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] Embodiment 1,

[0030] This embodiment provides a combined filter multispectral imaging sensor, the imaging sensor comprising an imaging sensor, an optimized multispectral filter array and an optimized liquid crystal tunable filter;

[0031] like Figure 1 As shown, the optimized multi-spectral filter array is installed in front of the imaging sensor target surface, and a circular light-transmitting hole is set in the central area of ​​the optimized multi-spectral filter array, and the optimized liquid crystal tunable filter is set in the circular light-transmitting hole.

[0032] The size of the optimized multi-spectral filter array is the same as the size of the imaging sensor target surface; Figure 2 As shown, the optimization of the multi-spectral filter array is as follows: the high-reflection film stack 1 (the first Bragg reflector) and the spacer layer of the Fabry-Perot interference cavity are coated with a coating device, and then the spacer layer is etched and changed by combining the photolithography mask technology and the ion beam etching technology, so as to change the thickness of the spacer layer to adjust the passband peak position of different spectral channels. By rationally designing the photolithography mask plate, a series of superimposed etchings (i.e., combined etching) of different thicknesses are performed on the spacer layer through overlay to obtain an array of spacer layers of different thicknesses, and finally, the high-reflection film stack 2 (the second Bragg reflector) of the Fabry-Perot interference cavity is coated to complete the multi-spectral filter coating, specifically: according to the classic formula of the Fabry-Perot filter: Optimize the spacer layer: change the wavelength transmitted by changing the thickness of the spacer layer, where λ represents the wavelength transmitted through the multi-spectral filter array, n represents the refractive index of the filter, d represents the thickness of the spacer layer, and k represents the wave number of the incident light. Φ1 and Φ2 represent the phase changes of the two surfaces of the filter, and m represents the interference order of the incident light; wherein the spacer layer is also a filter in the multi-spectral filter array.

[0033] The optimization of the liquid crystal tunable filter in this embodiment is to select a Lyot structure as the basic structure of the liquid crystal tunable filter, such as Figure 3 As shown, a two-layer cascaded Lyot structure is used instead of a multi-layer cascade. The transmission can be tuned to meet the central wavelength of the 450-900nm band according to the tuning range of the liquid crystal phase delay. The relationship between the first-order transmittance and the wavelength can be expressed as:

[0034]

[0035] Among them, δ1 is the phase difference of the first order, Δn1 is the birefringence difference of the first layer of optical material, and w1 is the thickness of the first layer of optical material. The subsequent i-th order transmittance generally satisfies the following relationship:

[0036]

[0037] The thickness of the liquid crystal box is designed to be 0.7mm (about 0.35mm for the upper and lower substrates), the thickness of the polarizer is 0.2mm, the liquid crystal material is nematic liquid crystal, and Δn (589nm.20℃) is about 0.26, which can avoid insufficient phase delay caused by the box thickness being too thin.

[0038] Embodiment 2,

[0039] This embodiment is a further limitation of Embodiment 1. The imaging method is implemented by using the above-mentioned combined filter multi-spectral imaging sensor. The method includes the following steps:

[0040] S1. The target background image is injected into the joint filter multi-spectral sensor to obtain 9 narrow band lights and the light to be tuned.

[0041] The relevant operations of step S1 are introduced with a specific example:

[0042] The optimized spectral filter array performs multi-spectral splitting on the spectral domain of the incident light. It is the core device and achieves the technical indicators required for the application of multi-spectral chip sensors by designing and optimizing the physical parameters of the pixel-level coating film system of the spectral modulation microstructure.

[0043] like Figure 1As shown, the light emitted by the detection target background image passes through the optical system and enters the joint filter. This embodiment uses a mosaic array with 3*3 multi-spectral channels, which are distributed around the joint filter, so that the optimized multi-spectral filter array can split light to obtain nine narrow band lights. The light source includes the light emitted by the detection target background image and the light emitted by the detection target. The light to be tuned is obtained by filtering the optimized liquid crystal tunable filter.

[0044] S2. Convert the narrow-band lights of 9 bands and the light to be tuned into image signals.

[0045] The relevant operations of step S2 are introduced with a specific example:

[0046] The narrow-band lights of 9 bands and the light to be tuned are emitted to the imaging optical path to obtain spectral cube data and spectral images; the imaging optical path is specifically as follows: the imaging sensor target surface converts the received light signal into an electrical signal and transmits it to a computer, and the computer converts the electrical signal into an image signal.

[0047] S3, restore the three-dimensional spectral image, and roughly detect the combined filter multi-spectral sensor.

[0048] The relevant operations of step S3 are introduced with a specific example:

[0049] The filter transmission curve of optimized multispectral filter array imaging is often defined as a spectral sensitivity function. By controlling the transmittance of the incident light in different spectral bands, the main features of the spectral band can be obtained on the sensor. The main parameters of each spectral band involve the sampling range, the sampling center band, and the transmittance variance distribution. The optimized multispectral filter array snapshot sensor can capture video in real time, but only one band can be captured per pixel. The multispectral imaging system based on the optimized multispectral filter array can use a high-pass filter array instead of a bandpass filter. The spectral mixing introduced by the high-pass filter ensures the advantage of high light flux. The high-pass filter blocks light in a specific narrow range, which is different from the bandpass filter. Then, a prototype system was built according to the basic principle. The system captures the stitched image in a single snapshot and recovers the three-dimensional spectral image from the captured image through an efficient demosaicing algorithm.

[0050] The three-dimensional spectral image restoration is specifically as follows: the grayscale value missing from the spectral cube data is restored by an inversion algorithm, specifically as follows: firstly, the shallow features of the mosaic image are extracted using a spatial channel attention model, then a convolutional neural network with an output channel of 9 is used to obtain a multi-spectral feature image mapped to a nine-band, and a long jump connection is added to add the shallow features to the inverted nine-band multi-spectral feature image, such as Figure 2As shown, a three-dimensional spectral image (the final demosaiced multispectral image) is obtained; the method of the coarse detection combined with the filter multispectral sensor is specifically as follows: the output three-dimensional spectral image is input to the computer terminal, and the average grayscale value of the nine bands in the three-dimensional spectral image is calculated, such as Figure 3 As shown, the bands with grayscale values ​​higher than the average grayscale value are deleted, and the target bands of this embodiment: band 6, band 7, band 8 and band 9 are retained to achieve rough detection.

[0051] S4, fine-tuning combined filter multispectral sensor.

[0052] The relevant operations of step S4 are introduced with a specific example:

[0053] The method for finely adjusting the combined filter multi-spectral sensor is specifically as follows: according to the target band, the control parameters of the optimized liquid crystal tunable filter are adjusted, and the adjustment accuracy is set to 10nm, so that the filter band matches the target band, and the finely adjusted combined filter multi-spectral sensor is obtained.

[0054] S5. Spectral imaging.

[0055] The relevant operations of step S5 are introduced with a specific example:

[0056] The light emitted by the detection target is emitted into the precisely adjusted combined filter multi-spectral sensor, the light of the target background image is filtered out, and the light to be imaged is obtained. The light to be imaged is emitted to the imaging light path to obtain spectral imaging.

Claims

1. A combined filter multispectral imaging sensor, characterized in that: include: Imaging sensor, optimized multi-spectral filter array, and optimized liquid crystal tunable filter; The optimized multi-spectral filter array is installed in front of the imaging sensor target surface, and a circular light-transmitting hole is set in the central area of ​​the optimized multi-spectral filter array, and the optimized liquid crystal tunable filter is set in the circular light-transmitting hole.

2. The combined filter multispectral imaging sensor according to claim 1, characterized in that: The size of the optimized multi-spectral filter array is the same as the size of the imaging sensor target surface; the optimization of the multi-spectral filter array is specifically as follows: Optimize the spacer layer: change the transmitted wavelength by changing the thickness of the spacer layer, where λ represents the wavelength transmitted through the multi-spectral filter array, n represents the refractive index of the filter, d represents the thickness of the spacer layer, k represents the wave number of the incident light, Φ1 and Φ2 represent the phase changes of the two surfaces of the filter, and m represents the interference order of the incident light.

3. The combined filter multispectral imaging sensor according to claim 1, characterized in that: The optimization of the liquid crystal tunable filter is specifically as follows: a Lyot structure is selected as the basic structure of the liquid crystal tunable filter, a two-layer cascaded Lyot structure is used instead of a multi-layer cascade, and the thickness of the control liquid crystal box is set to 0.7 mm.

4. A combined filter multispectral imaging method, characterized in that: The imaging method is implemented by using the combined filter multispectral imaging sensor according to any one of claims 1 to 3, and the method specifically comprises: S1, the target background image is injected into the joint filter multi-spectral sensor to obtain N bands of narrow band light and light to be tuned; S2, converting the narrow-band lights of N bands and the light to be tuned into image signals; S3, restore the three-dimensional spectral image, coarse detection combined with filter multi-spectral sensor; S4, fine-tuning combined filter multispectral sensor; S5. Spectral imaging.

5. The combined filter multispectral imaging method according to claim 4, characterized in that: The narrow-band lights of the N wavebands are obtained by splitting them by an optimized multi-spectral filter array; and the light to be tuned is obtained by filtering by an optimized liquid crystal tunable filter.

6. The combined filter multispectral imaging method according to claim 5, characterized in that: The method of converting the narrow-band light of N bands and the light to be tuned into the image signal specifically includes: the narrow-band light of N bands and the light to be tuned are emitted to the imaging optical path to obtain spectral cube data and spectral image; the imaging optical path specifically includes: the imaging sensor target surface converts the received light signal into an electrical signal and transmits it to a computer, and the computer converts the electrical signal into an image signal.

7. The combined filter multispectral imaging method according to claim 6, characterized in that: The specific method of restoring the three-dimensional spectral image is: restoring the missing grayscale values ​​of the spectral cube data through an inversion algorithm to obtain a three-dimensional spectral image; the specific method of the coarse detection combined with the filter multi-spectral sensor is: calculating the average grayscale value of N bands in the three-dimensional spectral image, deleting the bands with grayscale values ​​higher than the average grayscale value, and obtaining the target band.

8. The combined filter multispectral imaging method according to claim 7, characterized in that: The method for finely adjusting the combined filter multi-spectral sensor is specifically as follows: according to the target band, the control parameters of the optimized liquid crystal tunable filter are adjusted to make the filter band match the target band, thereby obtaining the finely adjusted combined filter multi-spectral sensor.

9. The combined filter multi-spectral imaging method according to claim 8, wherein the spectral imaging is specifically as follows: light emitted by the detection target is emitted into a precisely adjusted combined filter multi-spectral sensor, light of the target background image is filtered out, and light to be imaged is obtained, and the light to be imaged is emitted into an imaging optical path to obtain spectral imaging.

Images