A hyperspectral video imaging system
By using a joint design of front objective lens, spectroscopic prism, field integral element and spectroscopic imaging component in hyperspectral video imaging system, the problems of insufficient spectroscopic imaging performance and poor integration in existing systems in light and small platform applications are solved. High-resolution, fast shooting speed and strong anti-interference ability are achieved.
Patent Information
- Application Number
- CN202310269211.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2023-02-22
- Filing Date
- 2023-03-20
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2043-03-20
AI Technical Summary
The existing hyperspectral video imaging system has problems such as insufficient spectral imaging performance, low spatial and spectral resolution, slow shooting speed, weak anti-interference capability in light platforms or handheld applications such as drones and unmanned vehicles, and the system is integrated. The degree is poor, making it difficult to achieve optimal imaging performance.
Using a joint design of front objective lens, spectroscopic prism, field integral element and spectroscopic imaging component, the light distribution of transmission and reflection paths is imaged to a full-color focal plane detector and a multi-spectral focal plane detector respectively to achieve high spatial resolution Independent sampling and fusion of rate and hyperspectral resolution images, the spectroscopic imaging performance is improved by using collimating mirror groups, dispersing element groups and focusing mirror groups.
It realizes the high resolution, fast shooting speed and strong anti-interference ability of the hyperspectral video imaging system, and does not have parallax and distortion when image fusion, which improves the system's integration and application flexibility.
Smart Images

Figure CN116448246B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of hyperspectral imaging, and in particular to a hyperspectral video imaging system. Background Art
[0002] Hyperspectral imaging technology is an emerging field that combines spatial imaging technology and spectral technology. It is widely used in the fields of earth remote sensing, agriculture and forestry, mineral exploration, biomedicine, etc. Snapshot hyperspectral imaging technology can obtain the two-dimensional spatial information and one-dimensional spectral information of the target through one frame shooting, and then realize hyperspectral video imaging through near-real-time spectral restoration algorithm processing. It can be used in small and light platforms such as drones and unmanned vehicles or handheld application scenarios.
[0003] The Chinese patent publication number CN205539913U discloses a video hyperspectral camera, which consists of two imaging optical paths. An RGB camera is set on one path to perform secondary imaging on the primary image of the scene target formed by the telescope objective to obtain a color image with high spatial resolution. On the other path, a dichroic prism and a mask are added in front of the full-color camera. Through mask downsampling and prism splitting, the full-color camera obtains a low-spatial-resolution hyperspectral image of the target scene, and then through image fusion, a data cube with high spatial resolution and high spectral resolution is obtained. This solution collects and fuses high-spatial-resolution images and hyperspectral data separately through two optical systems, and can also achieve a high frame rate. However, the color image collected by the RGB camera in the two optical paths is obtained after secondary imaging, and the relative position relationship of the spatial sampling points in the hyperspectral image collected by the full-color camera cannot be kept completely consistent. The alignment of the color image and the hyperspectral image is difficult. The components used in the optical path of the system are all shelf products and are placed separately, with poor integration, and cannot achieve optimal imaging performance. Summary of the invention
[0004] The technical problem to be solved by the present invention is to overcome the deficiencies in the prior art and provide a hyperspectral video imaging system with excellent spectral imaging performance, high spatial and spectral resolution, fast shooting speed, and strong anti-interference ability. The system is suitable for light and small platforms such as drones, unmanned vehicles, and unmanned boats, or portable handheld applications, thereby broadening the application scenarios of hyperspectral imaging technology.
[0005] According to the technical solution provided by the present invention, the hyperspectral video imaging system includes a front objective lens and a spectroscopic prism for dividing the light into a transmission path and a reflection path according to the incident direction of the light. One of the transmission path and the reflection path passes through the field integration element and the spectroscopic imaging component in sequence and is imaged to a full-color focal plane detector, and the other path is imaged to a multi-spectral focal plane detector.
[0006] In one embodiment of the present invention, the spectroscopic imaging assembly comprises a collimating lens group, a dispersing element group and a focusing lens group, and the light incident directions of the collimating lens group, the dispersing element group and the focusing lens group are arranged in sequence.
[0007] In one embodiment of the present invention, the dispersion element group includes two Amisi prisms and an aperture stop, and the two Amisi prisms are symmetrically arranged on both sides of the aperture stop.
[0008] In one embodiment of the present invention, the dispersion element group includes a catadioptric prism, and a light incident side and a light emitting side of the catadioptric prism are respectively connected to a transmission prism.
[0009] In one embodiment of the present invention, the dispersion element group includes a dispersion prism, and the dispersion prism is a single prism, a cemented prism or a catadioptric prism.
[0010] In one embodiment of the present invention, the light between the collimating lens group and the dispersing element group is the incident light, the light between the dispersing element group and the focusing lens group is the outgoing light, the incident light and the outgoing light are symmetrical about a symmetry plane, and the collimating lens group and the focusing lens group are symmetrical about the symmetry plane.
[0011] In one embodiment of the present invention, the front objective lens is a transmission objective lens, a reflection objective lens or a catadioptric objective lens.
[0012] In one embodiment of the present invention, the multi-spectral focal plane detector is an RGB detector or a mosaic filter array detector having multiple spectral channels.
[0013] In one embodiment of the present invention, the field integrating element is a pinhole array, a microlens array, an optical fiber array or a spatial light modulator.
[0014] In one embodiment of the present invention, the spectroscopic imaging assembly includes a collimating lens group and a focusing lens group, and the focal length ratio of the collimating lens group and the focusing lens group is 0.1-10.
[0015] The above technical solution of the present invention has the following advantages compared with the prior art:
[0016] 1. Realize a hyperspectral video imaging system. Through the joint design of the front lens, field integration element, and spectroscopic imaging components, an optical system with excellent spectroscopic imaging performance is obtained. Through independent sampling and image fusion of high spatial resolution multispectral images and low spatial resolution hyperspectral images, high spatial resolution and high spectral resolution images with high frame rate are obtained to realize hyperspectral video imaging.
[0017] 2. The multispectral optical path and the hyperspectral optical path share the same front lens and there is no secondary imaging. The obtained multispectral and hyperspectral two-dimensional spatial images only differ in resolution, without parallax, image size difference and distortion difference, which is very convenient for object point registration during image fusion.
[0018] 3. The multispectral information of each object point can be directly collected, and the full-resolution hyperspectral image is obtained by fusion of the hyperspectral data of some object points and the multispectral data of all object points, avoiding the inaccurate hyperspectral reconstruction caused by the complete loss of spectral information of some object points.
[0019] 4. Highly integrated modular design, while achieving a compact structure, the front lens, field integrator or dispersion prism can be replaced according to different application scenarios, with high application flexibility. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.
[0021] Figure 1 This is a light path diagram of a first embodiment of a hyperspectral video imaging system of the present invention;
[0022] Figure 2 It is a schematic diagram of a mid-field integration element in a first embodiment of a hyperspectral video imaging system of the present invention;
[0023] Figure 3 It is a schematic diagram of the dispersion spectrum collected by the panchromatic focal plane detector in the hyperspectral video imaging system of the present invention;
[0024] Figure 4 is an MTF curve diagram of the first embodiment of the hyperspectral video imaging system of the present invention;
[0025] Figure 5 This is a light path diagram of a second embodiment of a hyperspectral video imaging system of the present invention;
[0026] Figure 6 This is a light path diagram of the third embodiment of the hyperspectral video imaging system of the present invention.
[0027] Explanation of the reference numerals in the specification: 1-front objective lens; 2-beam-splitting prism; 3-multi-spectral focal plane detector; 4-first field integrating element; 5-first beam-splitting imaging assembly; 6-first collimating lens group; 7-first dispersion element group; 8-first focusing lens group; 9-aperture stop; 10-full-color focal plane detector; 11-light; 12-second field integrating element; 13-second collimating lens group; 14-second dispersion element group; 15-second focusing lens group; 16-second beam-splitting imaging assembly; 17-third field integrating element; 18-third beam-splitting imaging assembly; 19-third dispersion element group; 20-first transmission prism; 21-folding prism; 22-second transmission prism. DETAILED DESCRIPTION
[0028] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.
[0029] Reference Figure 1 As shown, in order to make the imaging system have excellent spectroscopic imaging performance and high spatial and spectral resolution, the present invention includes a front objective lens 1 and a spectroscopic prism 2 for dividing the light into a transmission path and a reflection path according to the incident direction of the light 11, one of the transmission path and the reflection path passes through the field integration element and the spectroscopic imaging component in sequence, and is imaged to the full-color focal plane detector 10, and the other path is imaged to the multi-spectral focal plane detector 3.
[0030] The spectroscopic imaging assembly includes a collimator lens group, a dispersion element group and a focusing lens group, and the incident directions of the light 11 of the collimator lens group, the dispersion element group and the focusing lens group are arranged in sequence. The front objective lens 1 is a transmission objective lens, a reflection objective lens or a catadioptric objective lens. The multi-spectral focal plane detector 3 is an RGB detector or a mosaic filter array detector with multiple spectral channels. The focal length ratio of the collimator lens group and the focusing lens group is 0.1 to 10.
[0031] The light 11 between the collimating lens group and the dispersing element group is the incident light, and the light 11 between the dispersing element group and the focusing lens group is the outgoing light. The incident light and the outgoing light are symmetrical about a symmetry plane, and the collimating lens group and the focusing lens group are symmetrical about the symmetry plane.
[0032] Specifically, the working principle of the hyperspectral video imaging system of the present invention is as follows: light 11 from the target scene is imaged after passing through the front objective lens 1, and in its imaging optical path, the light 11 is divided into a transmission path and a reflection path by the beam splitter prism 2. Taking the imaging of the reflected path light 11 to the multi-spectral focal plane detector 3 and the imaging of the transmitted path light 11 to the field integrating element as an example, the field integrating element discretely samples the imaging of the transmission path of the front objective lens 1, and the light 11 is transmitted into the beam splitting imaging component after discrete sampling, and is collimated by the collimating lens group, dispersed and split by the dispersion element group, and focused and imaged by the focusing lens group, and then the spectral data of the discrete sampling points are obtained at the panchromatic focal plane detector 10.
[0033] In the transmission path, the field integration element can be a device that can realize discrete sampling, such as a pinhole array, a microlens array, an optical fiber array or a spatial light modulator. Here, the field integration element can be a pinhole array as an example for explanation. Figure 2 As shown, Figure 2 is a schematic diagram of the small hole array. Figure 3 It is a schematic diagram of the dispersion spectrum collected by the full-color focal plane detector 10. Since the field integrating element performs array-type discrete sampling on the image plane of the front objective lens 1, there are certain gaps between the sampling points. The spectroscopic imaging component disperses the complex light input from each sampling point to the gaps between the sampling points, and the full-color focal plane detector 10 records it. The dispersion spectrum of each sampling point recorded is independent of each other and has no aliasing. The high-spectral data of each sampling point can be extracted through the spectral reconstruction algorithm to obtain high-spectral data of the scene target with low spatial resolution.
[0034] The system captures a frame, and a multispectral focal plane detector 3 acquires a multispectral image with high spatial resolution, and a panchromatic focal plane detector 10 acquires and reconstructs a hyperspectral image with low spatial resolution, and the two images are fused to obtain a hyperspectral image with high spatial resolution. The multispectral focal plane detector 3 and the panchromatic focal plane detector 10 are controlled by a circuit system to realize simultaneous exposure and acquisition at the same frame rate, and an image fusion algorithm is run within a frame period to realize hyperspectral video imaging. The multispectral optical path and the hyperspectral optical path share a front lens, and the obtained two-dimensional spatial image only has a different resolution, and there is no parallax, image size difference and distortion difference, which is very convenient for object point registration during image fusion. In addition, the multispectral information of each object point can be directly acquired, and the full-resolution hyperspectral image is obtained by fusion of the hyperspectral data of some object points and the multispectral data of all object points, thereby avoiding the inaccurate hyperspectral reconstruction caused by the complete loss of spectral information of some object points. Embodiment 1:
[0035] As Figure 1As shown, the present invention includes a front objective lens 1, a beam splitter prism 2, an RGB detector, a first field integrator element 4-pinhole array, a first beam splitter imaging component 5, and a full-color focal plane detector 10. The first beam splitter imaging component 5 includes a first collimator lens group 6, a first dispersion element group 7, a focusing lens group, and an aperture stop 9. Light 11 from the target scene is imaged after passing through the front objective lens 1. In its imaging optical path, the beam splitter prism 2 divides the light 11 into a transmission path and a reflection path, wherein the reflection path light 11 is imaged to the RGB detector, and the transmission path light 11 is imaged to the pinhole array. The pinhole array discretely samples the image formed by the transmission path of the front objective lens 1. After discrete sampling, the light 11 is transmitted into the first beam splitter imaging component 5, and after being collimated by the first collimator lens group 6, dispersed and split by the first dispersion prism, and focused and imaged by the first focusing lens group 8, the spectral data of discrete sampling points are obtained at the full-color focal plane array.
[0036] Figure 2 : is a schematic diagram of the pinhole array, and the figure is a schematic diagram of the dispersion spectrum collected by the full-color focal plane detector 10 at this time. Since the pinhole array performs array-type discrete sampling on the image plane of the front objective lens 1, there is a certain gap between the sampling points. The first spectroscopic imaging component 5 disperses the complex light input from each sampling point to the gap between the sampling points, and is recorded by the full-color focal plane detector 10. The recorded dispersion spectrum of each sampling point is independent of each other and has no aliasing. The high-spectral data of each sampling point can be extracted through the spectral reconstruction algorithm to obtain high-spectral data of the scene target with low spatial resolution.
[0037] The system captures a frame, collects an RGB color image with high spatial resolution by the RGB detector, collects and reconstructs a hyperspectral image with low spatial resolution by the panchromatic focal plane detector 10, and fuses the two images to obtain a hyperspectral image with high spatial resolution. The RGB detector and the panchromatic focal plane detector 10 are controlled by the circuit system to realize simultaneous exposure and image acquisition at the same frame rate, and run the image fusion algorithm within the frame period to realize hyperspectral video imaging.
[0038] The relevant indicators of the hyperspectral video imaging system provided in Example 1 are:
[0039] Spectral range: 400~1000nm;
[0040] System F number: 4.5;
[0041] Field of view: 30°
[0042] Front objective lens 1 focal length: 30mm;
[0043] Image resolution: 3 million;
[0044] Spectral resolution: 5nm.
[0045] The resolution of the RGB detector and the full-color focal plane detector 10 used in Example 1 are both 2000×1500, the ratio of the transmitted and reflected light intensities of the beam splitter 2 is 5:1, and the pinhole array includes 100×75 sampling points.
[0046] In Example 1, the front objective lens 1 adopts a transmissive double Gaussian objective lens, with a structure of 5 groups and 7 pieces; the first collimating lens group 6 and the first focusing lens group 8 in the first beam splitting imaging component 5 are symmetrically placed in the same optical system, with a structure of 4 groups and 4 pieces respectively. The optical system parameters are as follows:
[0047]
[0048] The first dispersion element group 7 in the first spectroscopic imaging component 5 is two groups of separate Amici prisms. Each group of Amici prisms is made of H-FK61 and H-LAK4L materials glued together, with vertex angles of 45.5° and 34° respectively, and a center thickness of 10 mm. The two groups of Amici prisms constitute a direct-view dispersion prism group.
[0049] The MTF curve of the hyperspectral video imaging system provided in this embodiment is as follows: Figure 4 As shown, it has excellent imaging quality, close to the diffraction limit. Embodiment 2:
[0050] like Figure 5 As shown, the hyperspectral video imaging system of the present invention comprises: a front objective lens 1, a beam splitter prism 2, an RGB detector, a second field integrator element 12-microlens array, a second beam splitter imaging component 16, and a full-color focal plane detector 10. The second beam splitter imaging component 16 comprises a second collimator lens group 13, a second dispersion element group 14, a second focusing lens group 15, and an aperture stop 9. The light 11 from the target scene is imaged after passing through the front objective lens 1, and the beam splitter prism 2 divides the light 11 into a transmission path and a reflection path in its imaging optical path, wherein the reflection path light 11 is imaged to the RGB detector, and the transmission path light 11 is imaged to the microlens array. The microlens array downsamples the image formed by the transmission path of the front objective lens 1, and each microlens can form an image point. The downsampled light 11 is transmitted into the second beam splitter imaging component 16, and after being collimated by the second collimator lens group 13, dispersed and split by the second dispersion element group 14, and focused and imaged by the second focusing lens group 15, the spectral data of the down-dispersion sampling point is obtained at the full-color focal plane array. The system captures a frame, collects an RGB color image with high spatial resolution by the RGB detector, collects and reconstructs a hyperspectral image with low spatial resolution by the panchromatic focal plane detector 10, and fuses the two images to obtain a hyperspectral image with high spatial resolution. The RGB detector and the panchromatic focal plane detector 10 are controlled by the circuit system to realize simultaneous exposure and image acquisition at the same frame rate, and run the image fusion algorithm within the frame period to realize hyperspectral video imaging.
[0051] The relevant indicators of the hyperspectral video imaging system provided in this embodiment are:
[0052] Spectral range: 400~950nm;
[0053] System F number: 4.5;
[0054] Field of view: 30°
[0055] Front objective lens 1 focal length: 30mm;
[0056] Image resolution: 12 million;
[0057] Spectral resolution: 2.5nm.
[0058] The resolution of the RGB detector and the full-color focal plane detector 10 used in the system are both 4000×3000, the ratio of the transmitted and reflected light intensity of the beam splitter 2 is 9:1, and the pinhole array contains 200×150 sampling points.
[0059] The front objective lens 1 in this embodiment is the same system as the front objective lens 1 in Embodiment 1.
[0060] The second collimating lens group 13 and the second focusing lens group 15 in the second light splitting imaging assembly 16 of this embodiment are the same optical system and are symmetrically arranged, respectively having a structure of 5 groups and 7 pieces, and the optical system parameters are as follows:
[0061]
[0062] The second dispersion element group 14 in the second spectroscopic imaging component 16 is two groups of separate Amici prisms. Each group of Amici prisms is made of H-LAF54 and H-ZF88 materials glued together, with vertex angles of 13.5° and 5.8° respectively, and center thicknesses of 6mm and 4mm respectively. After the light 11 passes through the two groups of Amici prisms, a 29° optical path turn is generated.
[0063] The hyperspectral video imaging system provided in Example 2 adopts a microlens array and has high energy utilization. Embodiment 3:
[0064] like Figure 6As shown, the hyperspectral video imaging system of the present invention includes: a front objective lens 1, a beam splitter prism 2, a multi-spectral focal plane detector 3, a third field integration element 17-pinhole array, a third beam splitter imaging component 18, and a full-color focal plane detector 10. The third beam splitter imaging component 18 includes a third collimator lens group, a third dispersion prism group, and a third focusing lens group. The light 11 from the target scene is imaged after passing through the front objective lens 1, and the beam splitter prism 2 divides the light 11 into a transmission path and a reflection path in its imaging light path, wherein the transmission path light 11 is imaged to the multi-spectral focal plane detector 3, and the reflection path light 11 is imaged to the pinhole array. The pinhole array discretely samples the image formed by the reflection path of the front objective lens 1, and the light 11 after discrete sampling is transmitted into the third beam splitter imaging component 18, and after being collimated by the third collimator lens group, dispersed and split by the third dispersion element group 19, and focused and imaged by the third focusing lens group, the spectral data of the discrete sampling points are obtained at the full-color focal plane array. The system captures a frame, and the multispectral focal plane detector 3 acquires a multispectral image with high spatial resolution, and the panchromatic focal plane detector 10 acquires and reconstructs a hyperspectral image with low spatial resolution, and the two images are fused to obtain a hyperspectral image with high spatial resolution. The multispectral focal plane detector 3 and the panchromatic focal plane detector 10 are controlled by the circuit system to realize simultaneous exposure and acquisition at the same frame rate, and the image fusion algorithm is run within the frame period to realize hyperspectral video imaging.
[0065] The relevant indicators of the hyperspectral video imaging system provided in Example 3 are:
[0066] Spectral range: 400~950nm;
[0067] System F number: 4.5;
[0068] Field of view: 30°
[0069] Front objective lens 1 focal length: 30mm;
[0070] Image resolution: 2 million;
[0071] Spectral resolution: 2.5nm.
[0072] The multispectral focal plane detector 3 used in the system is a mosaic filter array type multispectral detector with a total of 2 million pixels, where each 4×4 sub-unit constitutes a pixel array period, and each pixel array period contains 16 bands of narrow-band filters. Therefore, the multispectral detector can collect multispectral images with a spatial resolution of 2 million and 16 spectral channels. The resolution of the full-color focal plane detector 10 is 1600×1250, the ratio of the transmitted and reflected light intensity of the beam splitter 2 is 2:1, and the pinhole array contains 100×80 sampling points.
[0073] In Example 3, the front objective lens 1 is the same system as the front objective lens 1 in Example 1, and the third collimation system and the third focusing system in the third spectroscopic imaging component 18 are the same as the second collimation system and the second focusing system in Example 2.
[0074] In Example 3, the third dispersion element group 19 includes a catadioptric prism, and a transmission prism is connected to the side of the catadioptric prism where the light 11 is incident and the side where the light 11 is emitted, respectively. The third dispersion element group 19 in the third spectroscopic imaging component 18 is a three-glued catadioptric prism, which is glued together by a first transmission prism 20, a catadioptric prism 21, and a second transmission prism 22. The two transmission prisms have the same structure, made of H-LAF54, with a vertex angle of 23.5° and a center thickness of 5 mm. The catadioptric prism 21 is made of H-ZF88GT, with a vertex angle of 63.7° and a center thickness of 15 mm.
[0075] The hyperspectral video imaging system provided in Example 3 uses the third dispersion element group 19 to partially correct the nonlinearity of prism dispersion, so that the spectrum sampling interval is more uniform and the wavelength resolution difference is small.
[0076] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.
Claims
1. A hyperspectral video imaging system, characterized in that: According to the incident direction of light, it includes a front objective lens and a spectroscopic prism for dividing the light into a transmission path and a reflection path, one of the transmission path and the reflection path passes through a field integrator and a spectroscopic imaging component in sequence, and is imaged to a full-color focal plane detector, and the other path is imaged to a multi-spectral focal plane detector; the spectroscopic imaging component includes a collimator group, a dispersion element group and a focusing lens group, the dispersion element group includes two Amici prisms and an aperture diaphragm, and the two Amici prisms are symmetrically arranged on both sides of the aperture diaphragm; the light between the collimator group and the dispersion element group is the incident light, the light between the dispersion element group and the focusing lens group is the outgoing light, the incident light and the outgoing light are symmetrical about a symmetry plane, and the collimator group and the focusing lens group are symmetrical about the symmetry plane.
2. The hyperspectral video imaging system according to claim 1, characterized in that: The light incident directions of the collimating lens group, the dispersing element group and the focusing lens group are arranged in sequence.
3. The hyperspectral video imaging system according to claim 1, characterized in that: The front objective lens is a transmission objective lens, a reflection objective lens or a catadioptric objective lens.
4. The hyperspectral video imaging system according to claim 1, characterized in that: The multi-spectral focal plane detector is an RGB detector or a mosaic filter array detector with multiple spectral channels.
5. The hyperspectral video imaging system according to claim 1, characterized in that: The field integration element is a pinhole array, a microlens array, an optical fiber array or a spatial light modulator.
6. The hyperspectral video imaging system according to claim 1, characterized in that: The spectroscopic imaging component comprises a collimating lens group and a focusing lens group, and the focal length ratio of the collimating lens group and the focusing lens group is 0.1-10.
Citation Information
Patent Citations
Common optical path combined optical field spectral imaging method and device
CN103471715A
Snapshot microscopic hyperspectral imaging system and imaging method
CN108414086A
Spectral imaging method, system and device
CN114754868A
High spectrum camera of video
CN205539913U
Hyperspectral imaging systems
US20060072109A1
