Light path multiplexing type snapshot polarization spectrum three-dimensional combined imaging detection system

Abstract

The invention relates to an optical path multiplexing type snapshot polarization spectrum three-dimensional combined imaging detection system, which is based on an optical path multiplexing double-channel combined imaging structure and is divided into a polarization channel and a polarization spectrum channel through a main optical path splitting system. Wherein the polarization channel obtains a high-spatial-resolution polarization image of a target, the high-spatial-resolution polarization image can serve as input data of polarization three-dimensional data inversion of the target and can also serve as spatial prior information of polarization spectrum image reconstruction of the target, and definition and accuracy of polarization spectrum image reconstruction of the target are improved; a polarization spectrum channel adopts an optical path multiplexing type structure, the system structure complexity and the installation and adjustment difficulty are reduced, and an obtained polarization spectrum image can be used as input data for inversion of a polarization spectrum reflectivity curve of a target. The system can provide polarization, spectrum and three-dimensional combined imaging data of the target at the same time, and can effectively improve the target recognition precision and the target material classification accuracy in the aspect of material target classification in a complex background environment.

Description

Technical Field

[0001] The present invention relates to an optical path multiplexing type snapshot polarization spectroscopy three-dimensional joint imaging detection system, which is a novel five-dimensional joint optical imaging detection system combining programmable coded aperture compressive spectroscopy real-time imaging technology and visible light broad spectral band polarization three-dimensional real-time imaging technology. Background Art

[0002] With the rapid development of optoelectronic technology and the demand for integrated detection of multi-dimensional information of targets in the field of remote sensing, in recent years, a variety of new optical remote sensing instruments for detecting multi-dimensional data of targets have emerged. The optical imaging integrated detection system based on spectral detection and polarization detection is a typical representative instrument among them. This type of system can detect the two-dimensional plane geometric data of the target using the x, y two-dimensional coordinate plane information, spectral information, and polarization information, invert the physical and chemical characteristic parameters of the target, provide rich multi-dimensional cube data for complex target modeling, and thus provide an effective technical means for target classification and recognition. While improving its performance and practicality, this technology is expanding towards higher dimensions. If the depth information of the spatial z coordinate dimension is added to the system modulation process, it will bring the five-dimensional data acquisition ability of three-dimensional space-spectroscopy-polarization, which will surely push multi-dimensional optical imaging detection to a brand-new remote sensing application level.

[0003] In terms of spectral imaging, the compressive spectral imaging technology emerging in recent years is different from the traditional spectral imaging method. Through information coding and modulation means, effective spatial information and spectral information can be restored with fewer measurement results. It has low requirements for the platform and high stability, especially having great advantages in data transmission and storage. It is one of the hotspots of today's new spectral imaging technologies. However, the current compressive spectral technology is still in the principle verification stage, and its system performance indicators need to be improved. For example, the complexity of the system structure needs to be further reduced to reduce the difficulty of system design and system alignment. In addition, the spectral calculation accuracy and speed also need to be urgently broken through, which is the key to successfully promoting this type of system to the application level.

[0004] The current real-time polarization imaging detection technologies mainly include the sub-aperture type, sub-amplitude type, polarization prism type, and sub-focal plane type. Among them, the structure of the sub-focal plane polarization system is the simplest, which is more conducive to the effective integration of multi-channel multi-dimensional information systems. However, based on the working principle of polarization devices, polarization imaging will surely reduce the light energy transmittance of the system. How to optimize the optical path structure in the multi-dimensional detection system so that the spectral channels sensitive to light energy transmittance are not affected by polarization devices, and use the broad spectral band light intensity information as an effective compensation for the spatial resolution of the spectral channels is another difficulty in the optimized design of the multi-dimensional detection system.

[0005] With the development of machine vision, three-dimensional imaging has received increasing attention. Currently, three-dimensional imaging technologies in the field of remote sensing include binocular stereo vision, structured light three-dimensional imaging, lidar three-dimensional imaging, holographic three-dimensional imaging, and polarization three-dimensional imaging. Among them, polarization three-dimensional imaging technology, as a non-coherent imaging technology, is not affected by the propagation path and coherent error, is less sensitive to ambient light, has high spatial resolution, and can perform real-time imaging. In some cases, it has more advantages in three-dimensional imaging of complex targets than traditional three-dimensional imaging technologies. This technology uses the mapping relationship between the polarization characteristics of the reflected light from the surface of the same material target with basically known physical characteristics and the contour characteristics of the target surface to invert the depth information of the target. Therefore, its application range is limited due to the overly demanding requirements for the physical characteristic parameters of the target and the composition of the material, as well as the problem of singular solutions in depth information inversion.

[0006] In summary, based on the urgent needs of China's aerospace remote sensing, marine remote sensing, and transportation fields for multi-dimensional real-time optical imaging technology, it is very necessary to design a new type of five-dimensional joint optical imaging detection system that combines a compact programmable coded aperture compressive spectroscopy real-time imaging technology and a visible light wide spectral band polarization three-dimensional real-time imaging technology to solve the inherent problems existing in the combination of snapshot coded aperture spectral imaging technology and real-time polarization detection technology. Summary of the Invention

[0007] The object of the present invention is to provide a light path multiplexing type snapshot polarization spectroscopy three-dimensional joint imaging detection system. This imaging system uses a two-channel imaging structure with light path multiplexing and a split focal plane polarization imaging technology, reduces the structural complexity of the traditional snapshot compressive imaging system, reduces the system alignment difficulty, increases the three-dimensional imaging ability and the calculation accuracy of the spectral image, and has five-dimensional imaging detection ability.

[0008] The system includes the following components:

[0009] Objective lens: Located at the very front of the entire imaging system, it is used to image the scene target within the field of view on its focal plane;

[0010] Beam splitting prism A: Located behind the objective lens, it splits the incident light focused by the objective lens into a polarization channel and a polarization spectroscopy channel;

[0011] Beam splitting prism B: Located behind the transmitted light of beam splitting prism A. In the outgoing light path, it transmits the incident light in the polarization spectroscopy channel into the light path multiplexing structure. In the return light path, it reflects the returned beam to the converging lens B;

[0012] Collimator: Located after the transmitted direction of the incident direction of beam splitting prism B, its object-side focal plane coincides with the image-side focal plane of the objective lens, collimates the incident light, and makes the light exit parallel according to a certain field of view and aperture;

[0013] Blazed grating: Located behind the collimating mirror, in the outgoing optical path, it disperses and spectroscopically analyzes the parallel light of each field of view incident thereon. In the return optical path, it is located on the image-side focal plane of the converging mirror A, used to eliminate the aberration caused by the dispersion and spectroscopic analysis in the outgoing optical path, and combine the light rays of each modulated spectral segment;

[0014] Converging mirror A: Located behind the blazed grating, the blazed grating is located on its image-side focal plane, and its object-side focal plane coincides with the position of the DMD, so that the light rays after dispersion and spectroscopic analysis are imaged on the DMD;

[0015] DMD: Located on the image-side focal plane of the converging mirror A, composed of an array of reflective micromirrors. By programming to load the corresponding coding matrix for it, the driving voltage of each micromirror can be controlled, so that each micromirror flips along its respective diagonal, thereby realizing the gating or blocking of the incident optical signal on each micromirror, and completing the coding modulation of the spectral dimension of the colored light;

[0016] Converging mirror B: Located in the reflection direction of the other incident direction of the beam-splitting prism B, it images the coded and modulated spectral information reflected by the beam-splitting prism B onto the sub-focal plane polarization camera B. At this time, the imaging plane is the third-order image plane;

[0017] Sub-focal plane polarization camera A: Located behind the reflected light of the beam-splitting prism A, it is a camera that can capture the polarization information of different regions in an image in a single shot. By integrating a special micro-polarization filter array on the imaging sensor, the camera can record information such as the intensity, polarization angle, and degree of polarization of light at the same time. After passing through the polarization filter, the light is divided into four light rays with different polarization states, and different linearly polarized state images can be obtained;

[0018] Sub-focal plane polarization camera B: Located behind the converging mirror B, the colored light with spatial and spectral information after the DMD completes the coding template modulation, after being combined by the blazed grating and reflected and split by the beam-splitting prism B, is imaged on the third-order image plane, that is, the sub-focal plane type polarization camera in the polarization spectral channel obtains the spectral channel detection image;

[0019] Computer: Controls the loading of the coding matrix of the DMD through a computer program, collects the polarization image data of the polarization channel and the coded and modulated image data of the polarization spectral channel, performs relevant image processing, and finally obtains the restored target polarization spectral image and three-dimensional data.

[0020] The working principle of the present invention is as follows: After the target reflected light passes through the imaging objective lens, it becomes a dual-channel structure through the beam-splitting prism A: A part of the light passes through the polarization channel and is imaged on the split focal plane type polarization camera A to obtain the visible light broadband polarization image of the target; Another part of the light enters the polarization spectroscopy channel, is collimated by the collimating lens after being transmitted by the beam-splitting prism B, undergoes dispersion on the blazed grating to generate spectral separation, and then is secondarily imaged on the DMD through the imaging lens. The DMD is an array of reflective micromirrors. The micromirrors can be flipped along the diagonal under voltage control, and the gating or blocking of the unit incident light signal can be realized by programming each micromirror. Its response time is in the microsecond level, and the state of the micromirror can be locked at any time. Therefore, by loading the designed coding template on the DMD, the high-speed and continuous control of the light in the required wavelength band can be satisfied, and the programmable modulation coding operation in the spatial and spectral dimensions can be completed. After the DMD completes the coding template modulation, it reflects the colored light with spatial and spectral compression information. After passing through the reverse process of the aforementioned process, that is, combining light and collimation, it passes through the beam-splitting prism B for the second time, and after being reflected by it, it is imaged on the third image plane, that is, the split focal plane type polarization camera B in the polarization spectroscopy channel, to obtain the compressed polarization spectroscopy image. According to the different modes of DMD loading coding during acquisition, the polarization spectroscopy image is reconstructed through the restoration algorithm with the Hadamard transform, compressed sensing theory, and deep learning theory as the core. Finally, the polarization spectroscopy imaging subsystem obtains the image data of the target's two-dimensional space, one-dimensional spectrum, and one-dimensional polarization, and the visible broadband light polarization imaging subsystem obtains the z-coordinate depth data of the target's three-dimensional space.

[0021] The present invention has the following advantages compared with the existing polarization spectroscopy system:

[0022] The optical path multiplexing dual-channel optical structure of the system reduces the structural complexity of the traditional snapshot compression imaging system, reduces the system assembly and adjustment difficulty, and combines the compressed spectroscopy image reconstruction algorithm with the prior knowledge of two-dimensional space information to improve the accuracy of aliased spectrum resolution and reconstruction of multi-dimensional data cubes without sacrificing the system detection performance, effectively improving the target detection ability. At the same time, the polarization channel can detect three-dimensional polarization information, so the system has five-dimensional imaging detection ability. Brief Description of the Drawings

[0023] Figure 1 It is the schematic diagram of the system composition principle of the present invention;

[0024] Figure 2 It is the schematic diagram of the microarray and pixel arrangement of the split focal plane polarization camera of the present invention;

[0025] Figure 3 It is the binary random coding aperture template diagram of the present invention;

[0026] Figure 4 It is the multi-dimensional joint real-time optical imaging detection flow chart of the present invention;

[0027] The specific reference numerals in the figure are as follows:

[0028] 1. Objective lens; 2. Beam splitting prism A; 3. Beam splitting prism B

[0029] 4. Collimator; 5. Blazed grating; 6. Converging lens A

[0030] 7. DMD; 8. Converging lens B; 9. Split focal plane polarization camera B

[0031] 10. Split focal plane polarization camera A; 11. Computer Specific implementation manners

[0032] The technical solution of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0033] The present invention provides a snapshot polarization spectroscopy three-dimensional joint imaging detection system with optical path multiplexing. This imaging system utilizes a dual-channel imaging structure with optical path multiplexing and split focal plane polarization imaging technology. As Figure 1 , the present invention includes an objective lens 1, a beam splitting prism A 2, a beam splitting prism B 3, a collimator 4, a blazed grating 5, a converging lens A 6, a DMD 7, a converging lens B 8, a split focal plane polarization camera A 9, a split focal plane polarization camera B 10, and a computer 11. After the incident light of the scene target passes through the imaging objective lens 1, it becomes a dual-channel structure through the beam splitting prism A 2: a part of the light forms a primary image plane through the polarization channel on the split focal plane polarization camera A 10. As Figure 2 , after being modulated by the micro-polarization array in front of the CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, the linear polarization degrees of every four pixels detected by the camera are 0°, 45°, 90°, and 135° respectively, so as to obtain the polarization sub-images of the four linear polarization states in the visible light broadband spectrum of the target in real time; another part of the light enters the polarization spectroscopy channel. First, it is transmitted through the beam splitting prism B 3, and then the collimator 4 receives the transmitted light from the primary image plane and collimates it onto the surface of the blazed grating 5. Dispersion spectroscopy is performed on the blazed grating 5, and light of different wavelengths is separated and emitted at different angles, resulting in spectral separation. Then, it is secondarily imaged on the DMD 7 through the converging lens A 6. Corresponding codes, such as Hadamard, multi-slit, random codes, etc., are loaded on the DMD 7 according to the reconstruction method of the compressed spectral original image. Assuming that the system uses random coding, the computer 11 communicates with the DMD 7 through the DMD 7 drive circuit, and changes the coding matrix loaded on the DMD 7 to a binary Gaussian random code. As Figure 3, white represents gating where light can be reflected back along the original optical path, and black represents blocking where light is reflected in other directions. Because of the dispersion spectroscopy of the blazed grating 5, the DMD 7 can perform shift random coding modulation on images of different spectral channels, that is, a 2-pixel shift is generated horizontally between the random codings of each spectral channel (the number of shifted pixels is determined by the dispersion coefficient of the blazed grating 5, the element size of the DMD 7, and the pixel size of the split focal plane polarization camera B9). The image after coding modulation is reflected by the DMD 7, passes through the converging lens A6, and then undergoes spectral combination through the blazed grating 5 again to restore the originally dispersed light to the original light. Since the spectral lines of different spectral channels have undergone different coding modulations at this time, the spectral information after combination contains the spectral spatial modulation information of different spectral channels. Then, the collimating lens 4 collimates the light reflected back by the DMD 7, and finally, the light forms an image three times on the split focal plane polarization camera B9 through the beam splitting prism B3 and the converging lens B8. Its polarization modulation principle is the same as that of the split focal plane polarization camera A10. At this time, the information obtained is the polarization spectral information including spectral spatial coding modulation. The image information finally obtained by the dual-channel imaging system is respectively: the complete high-spatial-resolution polarization two-dimensional image information obtained from the polarization channel, and the coded and modulated polarization spectral information obtained from the polarization spectral channel. These two parts of information are transmitted into the computer 11 for subsequent algorithm processing of polarization spectral three-dimensional reconstruction. The information obtained by the two imaging channels in the imaging part can be used as a priori information for each other to improve the quality of information reconstruction. Specifically, it includes: 1) Using the high-spatial-resolution image of the target obtained by the visible light broadband polarization imaging subsystem as an additional input for polarization spectral imaging reconstruction to provide spatial a priori information and improve the clarity and accuracy of polarization spectral image reconstruction; 2) Using the polarization spectral reflectance curve of the target surface in the field of view obtained by polarization spectral imaging reconstruction to effectively classify the target material as a priori information for polarization three-dimensional reconstruction, and improve the possibility of polarization three-dimensional inversion when there are multiple unknown materials in the field of view. Finally, the polarization spectral imaging subsystem obtains the image data of the target's two-dimensional space, one-dimensional spectrum, and one-dimensional polarization, and the visible broadband light polarization imaging subsystem obtains the z-coordinate depth data of the target's three-dimensional space, completing the polarization spectral three-dimensional joint imaging detection.

[0034] The working flowchart of the present invention is as Figure 4As shown in the figure. First, select the detection target, determine the altitude angle and azimuth angle of the light source, adjust the detector parameters, and obtain the complete target image data I(x, y) with high spatial resolution in the polarization channel; obtain the encoded and modulated polarization spectral image f(x, y, λ)T(x, y) in the polarization spectral channel, where T(x, y) is the binary random encoding matrix loaded by the DMD. Utilize the complete target image I(x, y) with high spatial resolution to improve the spectral resolution accuracy of the encoded and modulated image, thereby improving the reconstruction quality of the original polarization spectrum f(x, y, λ). After obtaining the original polarization spectral image, the polarization spectral reflectance data of the target can be calculated to provide additional input for the polarization three-dimensional data calculation network, thereby obtaining the polarization three-dimensional data f(x, y, z) of the target. Finally, change the shooting position according to the experimental requirements, shoot multiple groups of polarization spectral image data, and finally obtain the five-dimensional data f(x, y, z, λ, p) of the three-dimensional space-spectral-polarization of the scene target, thereby improving the reconstruction quality of the polarization spectral image and increasing the polarization three-dimensional detection ability.

[0035] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention. The content not detailedly described in the specification of the present invention belongs to the prior art well-known to those skilled in the art.

Claims

1. An optical path multiplexing snapshot polarization spectrum three-dimensional joint imaging detection system, consisting of an objective lens, a beam splitter prism A, a beam splitter prism B, a collimator, a blazed grating, a converging lens A, a digital micromirror device (DMD), a converging lens B, a focal plane polarization camera A, a focal plane polarization camera B and a computer, characterized in that: Includes the following parts: Objective lens: Located at the front end of the entire imaging system, it is used to image the scene target within the field of view on its focal plane; Beam splitter prism A: located behind the objective lens, it splits the incident light focused by the objective lens into polarization channel and polarization spectrum channel; Beam splitter prism B: located behind the transmitted light of beam splitter prism A, in the outgoing light path, it transmits the incident light of the polarization spectrum channel into the optical path multiplexing structure, and in the return light path, it reflects the returned light beam to the converging mirror B; Collimator: Located after the incident direction of the beam splitter prism B, its object focal plane coincides with the image focal plane of the objective lens, collimating the incident light so that the light is emitted in parallel according to a certain field of view and aperture; Blazed grating: located behind the collimator, it disperses and splits the incident parallel light of each field of view in the outgoing light path. In the return light path, it is located on the image focal plane of the converging lens A to eliminate the aberration caused by the dispersion and splitting in the outgoing light path, and combine the modulated light of each spectrum segment; Converging lens A: located behind the blazed grating, the blazed grating is located on its image focal plane, and its object focal plane coincides with the position of the DMD, so that the light after dispersion and splitting is imaged on the DMD; DMD: Located on the focal plane of the image side of the converging mirror A, it is composed of a reflective micromirror array. By programming and loading the corresponding coding matrix for it, the driving voltage of each micromirror can be controlled to make each micromirror flip along its own diagonal line, thereby realizing the gating or blocking of the incident light signal on each micromirror, and completing the coding modulation of the spectral dimension of the dispersed light; Converging lens B: located in the reflection direction of the other incident direction of beam splitter prism B, imaging the coded modulated spectral information reflected back from beam splitter prism B to the focal plane polarization camera B. At this time, the imaging plane is a cubic image plane; Focus plane polarization camera A: Located behind the reflected light from beam splitter prism A, it is a camera that can capture polarization information of different areas in an image in a single shot. The camera integrates a special micro polarization filter array on the imaging sensor to simultaneously record information such as the intensity, polarization angle, and degree of polarization of the light. After passing through the polarization filter, the light is separated into four different polarization states, and images of different linear polarization states can be obtained; Focus plane polarization camera B: Located behind the converging mirror B, the dispersed light with spatial and spectral information after DMD completes the modulation of the coding template is imaged on the third image plane, i.e. the focus plane polarization camera of the polarization spectrum channel, to obtain the spectral channel detection image; Computer: The coding matrix loaded with DMD is controlled by a computer program, the polarization image data of the polarization channel and the coded modulation image data of the polarization spectrum channel are collected, and relevant image processing is performed to finally obtain the restored target polarization spectrum image and three-dimensional data.

2. The optical path multiplexing snapshot polarization spectrum three-dimensional joint imaging detection system according to claim 1, characterized in that: The dual-channel joint imaging structure based on optical path multiplexing includes a main optical path splitting system, a visible light wide-band polarization imaging subsystem and a polarization spectrum imaging subsystem. The main optical path splitting system divides the system into a polarization channel and a polarization spectrum channel. The polarization high spatial resolution image collected by the visible spectrum polarization imaging subsystem first provides the original data for polarization three-dimensional data inversion, and then serves as an additional input for polarization spectrum imaging reconstruction to provide spatial prior information, thereby improving the clarity and accuracy of polarization spectrum image reconstruction. The polarization spectrum imaging subsystem adopts an optical path multiplexing structure to reduce the complexity of the system structure and the difficulty of installation and adjustment. The polarization spectrum reflectance curve of the target surface in the field of view obtained by polarization spectrum imaging reconstruction is used to effectively classify the target material, which is used as prior information for polarization three-dimensional reconstruction to invert the polarization three-dimensional data of various unknown materials in the field of view. The system effectively combines the compressed polarization spectrum real-time imaging technology of the programmable spatial light modulator and the visible light wide-band real-time polarization imaging technology based on the focal plane to realize the joint imaging detection of the target three-dimensional space-spectrum-polarization five-dimensional information.

Images