Push-broom hyperspectral imaging system for unmanned aerial vehicle

Abstract

The invention relates to a push-broom hyperspectral imaging system for an unmanned aerial vehicle, and the system is characterized in that a line-plane combined diaphragm is provided with a slit and a rectangular window, a front lens images a target scene to the rectangular window, the target scene is collimated by a collimating lens to generate parallel light, the parallel light does not undergo dispersion after passing through a dispersion prism, and finally the parallel light reaches a focal plane detector through a focusing lens; high-resolution images are obtained, and pose change information is obtained by comparing the high-resolution images between adjacent frames; the front lens is also used for imaging a target scene to the slit, the target scene is collimated by the collimating lens to generate parallel light, the parallel light is dispersed by the dispersing prism, and finally the parallel light reaches the focal plane detector through the focusing lens to obtain a multispectral image; and calibrating the multispectral image of the current frame by using the pose change information, and splicing after calibration to obtain a final hyperspectral image of the target scene. The system is simple in structure and easy to use, and does not need an additional system to assist in image splicing.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral imaging, and in particular to a push-broom hyperspectral imaging system for unmanned aerial vehicles. Background Art

[0002] In the 1980s, with the development of earth observation applications, imaging spectroscopy technology emerged. It is an emerging field that combines space imaging technology and spectral imaging technology, which has greatly broadened the scope of application in aerospace remote sensing, agricultural and forestry resource detection, mineral resources and geological exploration, military detection, biomedicine and other fields.

[0003] As drone technology becomes increasingly mature, the new push-broom hyperspectral imaging technology based on light and small drone platforms has been favored by remote sensing workers. It makes the originally difficult, time-consuming and costly remote sensing data collection work simple, fast and low-cost.

[0004] Nowadays, the push-broom hyperspectral imaging system carried by drones has demonstrated strong application potential and market value in agriculture, water environment monitoring, forestry management, disaster relief, aerospace and other fields due to its efficient and accurate data collection capabilities.

[0005] Push-broom hyperspectral imaging technology can only obtain hyperspectral data of a strip of the ground at the same time, and a complete data cube of the target area is obtained after the drone flies over the ground. This type of system can obtain data with high spatial resolution and high spectral resolution, and has been widely used on satellite platforms.

[0006] The primary problem in extending this technology to UAV platforms is the stability of UAVs. Light and small UAVs operating in the troposphere are easily affected by factors such as airflow and vibration. It is difficult for their speed and attitude to remain consistent during flight, resulting in misalignment or loss of ground sampling by the hyperspectral imaging system during push-brooming. This unstable sampling will cause serious distortion of the geometric image after the strips are spliced. At present, the geometric calibration methods for push-broom hyperspectral images are mainly focused on post-processing of data based on POS data or auxiliary array cameras. For example, the team led by Xue Qingsheng of Ocean University of China proposed an optical system with a common objective lens for hyperspectral imaging and array imaging. The low-precision POS (Position and Orientation System) data obtained by consumer-grade positioning and inertial navigation devices (GPS / IMU) and auxiliary array cameras are used to perform geometric calibration of push-broom hyperspectral images. First, the low-precision POS data is used to perform preliminary calibration of the push-broom strips, and then high-precision relative POS information is extracted from the synchronously acquired inter-frame array images for geometric alignment of hyperspectral images. The teams of Zhao Chunjiang and Yang Guijun from Beijing Agricultural Information Technology Research Center, Jurado from University of Jaén, Spain, Li Haiwei from Xi'an Jiaotong University, and VVShipko from Special Instruments Science and Technology Center of Russian Academy of Sciences used independent array cameras to provide high-precision POS data to assist in the geometric calibration of hyperspectral images. The team of Zhang Hao from Institute of Remote Sensing and Digital Earth of Chinese Academy of Sciences only used low-precision POS information, and used SIFT and RANSAC algorithms to perform geometric calibration on scanned images. He also pointed out that using additional RGB images to assist would yield higher-precision calibration results.

[0007] In summary, the above solutions mainly assist image stitching by adding additional systems, such as using POS systems or array cameras to obtain posture change information for image correction, which can neither guarantee synchronization nor increase weight costs. Therefore, how to achieve anti-interference acquisition of hyperspectral images while ensuring synchronization and reducing weight costs as much as possible is an urgent problem to be solved in the push-broom hyperspectral imaging system of unmanned aerial vehicles. Summary of the invention

[0008] To this end, the technical problem to be solved by the present invention is to overcome the problem that the push-broom hyperspectral imaging system in the prior art needs to add an additional system (such as a POS system or an area array camera) to provide posture change information to assist in image stitching correction, thereby failing to ensure synchronization and also increasing the weight.

[0009] In order to solve the above technical problems, the present invention provides a push-broom hyperspectral imaging system for unmanned aerial vehicles, comprising a front lens, a line-surface combination aperture, a collimating lens, a dispersion prism, a focusing lens and a focal plane detector arranged in sequence, wherein the line-surface combination aperture is provided with a slit and a rectangular window, wherein:

[0010] The front lens images the target scene into a rectangular window, collimates it through a collimating lens to generate parallel light, passes through a dispersion prism without dispersion, and finally reaches a focal plane detector through a focusing lens to obtain a high-resolution image. By comparing the high-resolution images between the current frame and the previous frame, the posture change information is calculated;

[0011] At the same time, the front lens also images the target scene to the slit, which is collimated by the collimating lens to generate parallel light, which is then dispersed by the dispersion prism and finally reaches the focal plane detector through the focusing lens to obtain a multi-spectral image;

[0012] The multispectral image of the current frame is calibrated using the posture change information, and the calibrated multispectral images are spliced ​​to obtain a final hyperspectral image of the target scene.

[0013] In one embodiment of the present invention, a wide-band high-transmittance optical plate is provided at the light output of the slit, and a narrow-band filter is provided at the light output of the rectangular window.

[0014] In one embodiment of the present invention, the broadband high-transmittance optical plate and the narrow-band filter have the same thickness, so that the light emitted through the slit and the rectangular window has the same optical path difference.

[0015] In one embodiment of the present invention, the bandwidth of the narrowband filter is less than or equal to the spectral resolution of the hyperspectral imaging system, so as to prevent the light emitted from the rectangular window from forming a multi-spectral image on the focal plane detector.

[0016] In one embodiment of the present invention, the slit and the rectangular window are arranged in parallel in the length direction, and the distance between the slit and the rectangular window ranges from 1 mm to 3 mm.

[0017] In one embodiment of the present invention, the distance between the front lens and the line-plane combined aperture is equal to the focal length of the front lens, the distance between the line-plane combined aperture and the collimating lens is equal to the focal length of the collimating lens, and the distance between the focusing lens and the focal plane detector is equal to the focal length of the focusing lens.

[0018] In one embodiment of the present invention, the distance between the collimating lens and the dispersion prism ranges from 1 mm to 2 mm.

[0019] In one embodiment of the present invention, the distance between the dispersion prism and the focusing lens ranges from 1 mm to 2 mm.

[0020] In one embodiment of the present invention, the size of the rectangular window is 10 mm*5 mm, and the size of the slit is 10 mm*10 μm.

[0021] In one embodiment of the present invention, the front lens, line-surface combined aperture, collimating lens, dispersion prism, focusing lens and focal plane detector are coaxially arranged.

[0022] The above technical solution of the present invention has the following advantages compared with the prior art:

[0023] The push-broom hyperspectral imaging system for unmanned aerial vehicles of the present invention constructs a line-surface combined aperture, which is provided with a slit and a rectangular window. Hyperspectral information is obtained from a multispectral image formed by an object plane passing through the slit, and posture change information is obtained from a high-resolution image formed by an object plane passing through the rectangular window. There is no need to add an additional array camera to provide images and obtain posture change information therefrom, and there is no need for an independent GPS / I MU to provide POS data. The present invention is simple and effective.

[0024] The present invention effectively reduces the weight of the hyperspectral imaging system, ensures synchronization, and has a simple overall structure, which is suitable for large-scale promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.

[0026] Figure 1 Schematic diagram of the structure of a push-broom hyperspectral imaging system for a UAV in an embodiment of the present invention;

[0027] Figure 2 is a first viewing angle schematic diagram of a line-surface combined aperture in an embodiment of the present invention;

[0028] Figure 3 is a second viewing angle schematic diagram of a line-surface combined aperture in an embodiment of the present invention;

[0029] Figure 4 It is a side view of the line-surface combined aperture in an embodiment of the present invention.

[0030] Explanation of the reference numerals in the specification: 1. front lens; 2. line-surface combined aperture; 2.1. slit; 2.2. rectangular window; 2.3. wide-band high-transmittance optical plate; 2.4. narrow-band filter; 3. collimating lens; 4. dispersion prism; 5. focusing lens; 6. focal plane detector; 7. multispectral image; 8. high-resolution image; 9. computer. DETAILED DESCRIPTION

[0031] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0032] Embodiment 1

[0033] Reference Figure 1 As shown, the present invention relates to a push-broom hyperspectral imaging system for unmanned aerial vehicles, comprising a front lens 1, a line-surface combined aperture 2, a collimating lens 3, a dispersion prism 4, a focusing lens 5 and a focal plane detector 6 which are arranged in sequence, and these optical hardware need to be coaxially placed.

[0034] Furthermore, the focal plane detector 6 of this embodiment is also connected to a computer 9, and the computer 9 is used to receive data from the focal plane detector 6 and obtain the final hyperspectral image of the target scene through calculation.

[0035] See also Figure 2 , a line-surface combined aperture 2 is provided with a slit 2.1 and a rectangular window 2.2, wherein the front lens 1 images the target scene to the rectangular window 2.2, which is collimated by the collimating lens 3 to generate parallel light, which is then dispersed by the dispersion prism 4, and finally reaches the focal plane detector 6 through the focusing lens 5 to obtain a high-resolution image 8, and the posture change information is calculated by comparing the high-resolution image 8 between the current frame and the previous frame; at the same time, the front lens 1 also images the target scene to the slit 2.1, which is collimated by the collimating lens 3 to generate parallel light, which is then dispersed by the dispersion prism 4, and finally reaches the focal plane detector 6 through the focusing lens 5 to obtain a multispectral image 7; the multispectral image 7 of the current frame is calibrated using the posture change information, and the calibrated multispectral image 7 is spliced ​​to obtain the final target scene hyperspectral image. It should be noted that the optical plate 2.3 corresponding to the slit 2.1 does not filter light, so the light passing through the optical plate 2.3 will be dispersed; while the narrowband filter 2.4 corresponding to the rectangular window 2.2 filters light, so the light passing through the narrowband filter 2.4 will not be dispersed. The optical plate 2.3 and the narrowband filter 2.4 are introduced in detail below.

[0036] See also Figure 3 and Figure 4, a wide-band high-transmittance optical plate 2.3 is provided at the light output of the slit 2.1, and the wide-band high-transmittance optical plate 2.3 is used to combine with subsequent optical elements to provide a multi-spectral image 7; a narrow-band filter 2.4 is provided at the light output of the rectangular window 2.2, and the narrow-band filter 2.4 is used to combine with subsequent optical elements to provide a high-resolution image 8, and the posture change information is obtained through the high-resolution image 8 between adjacent frames. It should be noted that according to Figure 2 and Figure 3 It is not difficult to find that the slit 2.1 and the rectangular window 2.2 are both hollowed out on the line-surface combination aperture 2. In this embodiment, the side of the line-surface combination aperture 2 facing the front lens 1 is the side of incident light, and the side of the line-surface combination aperture 2 facing the collimating lens 3 is the side of outgoing light, that is, the wide-band high-transmittance optical plate 2.3 is arranged on the side of the slit 2.1 for outgoing light, and the narrow-band filter 2.4 is arranged on the side of the rectangular window 2.2 for outgoing light.

[0037] The following is a detailed introduction to this embodiment:

[0038] The relevant indicators of the push-broom hyperspectral imaging system for UAVs of this embodiment are as follows:

[0039] Spectral range of hyperspectral imaging system: 400-900nm;

[0040] Spectral resolution of the hyperspectral imaging system: 5nm on average;

[0041] Narrowband filter 2.4 has a central wavelength of 550nm and a bandwidth of 5nm;

[0042] Narrowband filter 2.4 thickness: 0.5mm-1mm, preferably 0.5mm;

[0043] Narrowband filter 2.4 size: 10mm*0.5mm;

[0044] The thickness of the wide-band high-transmittance optical plate 2.3 is 0.5mm-1mm, which needs to be consistent with the thickness of the narrow-band filter 2.4, preferably 0.5mm;

[0045] Wide-band high-transmittance optical flat plate 2.3 Size: 10mm*5mm;

[0046] Distance between the rectangular window 2.2 and the slit 2.1: 1mm-3mm, preferably 2mm;

[0047] Rectangular window 2.2 size: 10mm*5mm;

[0048] Line-surface combined aperture 2 size: 10mm*12mm*0.5mm;

[0049] Slit 2.1 size: 10mm*10μm;

[0050] Focal plane detector 6 pixel size: 10μm×10μm;

[0051] Focal plane detector 6 size: 2560×1280 pixels;

[0052] Front lens 1 focal length: 40mm-50mm, preferably 40mm;

[0053] Collimating lens 3 focal length: 40mm-50mm, preferably 40mm;

[0054] Focusing lens 5 focal length: 40mm-50mm, preferably 40mm;

[0055] Dispersion prism 4: Direct-view Amisi prism;

[0056] The object side half field of view angle of the hyperspectral imaging system is 5.71°.

[0057] Furthermore, the front lens 1 in this embodiment is an image-side telecentric lens.

[0058] Furthermore, in the present embodiment, the length of the slit 2.1 is 10 mm, and the object-side field angle of the hyperspectral imaging system is calculated to be 13.25°-14.25°, preferably 14.25°. At the same time, the light passing through the front lens 1 needs to cover both the slit 2.1 and the rectangular window 2.2, and the diameter of the minimum circumscribed circle of the slit 2.1 and the rectangular window 2.2 is 12.21 mm. The object-side field angle of the front lens 1 is calculated to be 33°-34°, preferably 34°. It should be noted that the object-side field angle of the hyperspectral imaging system in the present embodiment needs to satisfy that the imaged object covers the slit 2.1, and the object-side field angle of the front lens 1 needs to satisfy that the imaged object covers the slit 2.1 and the rectangular window 2.2.

[0059] Furthermore, a preset distance (1mm-3mm) is left between the slit 2.1 and the rectangular window 2.2, and the preset distance is used to ensure that the high-resolution image 8 formed by the object plane through the rectangular window 2.2 and the multi-spectral image 7 formed by the object plane through the slit 2.1 will not be aliased.

[0060] refer to Figure 1 As for the indicators related to the push-broom hyperspectral imaging system for drones provided in this embodiment, the target scene needs to be imaged at the line-surface combination aperture 2 through the front lens 1, so the distance between the front lens 1 and the line-surface combination aperture 2 is the focal length of the front lens 1. Similarly, the distance between the line-surface combination aperture 2 and the collimating lens 3 is the focal length of the collimating lens 3.

[0061] After the light is collimated by the collimating lens 3, it becomes parallel light. Therefore, the distance between the collimating lens 3 and the dispersion prism 4 should be as small as possible to reduce the volume of the system. In this embodiment, it is set to 1mm-2mm, preferably 1mm. Similarly, after the light is dispersed by the dispersion prism 4, it is still parallel light. Therefore, the distance between the dispersion prism 4 and the focusing lens 5 should also be as small as possible. In this embodiment, it is set to 1mm-2mm, preferably 1mm. The dispersed light needs to pass through the focusing lens 5 to form a multi-spectral image 7 on the focal plane detector 6 to provide high-spectral information. The light that has not been dispersed needs to pass through the focusing lens 5 to form a high-resolution image 8 on the focal plane detector 6 to provide posture change information. Therefore, the distance between the focusing lens 5 and the focal plane detector 6 is the focal length of the focusing lens 5.

[0062] refer to Figure 2 and Figure 3 In this embodiment, the line-surface combined aperture 2 is provided with a slit 2.1 and a rectangular window 2.2. A narrowband filter 2.4 needs to be added behind the rectangular window 2.2 to prevent the light emitted from the rectangular window 2.2 from forming a multi-spectral image on the focal plane detector 6. Similarly, a wide-band high-transmittance optical plate 2.3 with the same thickness as the narrowband filter 2.4 needs to be added behind the slit 2.1 to prevent the light emitted from the slit 2.1 and the rectangular window 2.2 from having a difference in optical path difference. Specifically, in this embodiment, in order to prevent the light emitted from the rectangular window 2.2 from forming a multi-spectral image on the focal plane detector 6, the bandwidth (5nm) of the narrowband filter 2.4 is not greater than the spectral resolution (5nm) of the hyperspectral imaging system. The spectral resolution of the hyperspectral imaging system is determined by the pixel size of the slit 2.1 and the focal plane detector 6.

[0063] Furthermore, the light passing through the rectangular window 2.2 is filtered by a narrowband filter 2.4, and only the light with a central wavelength of 550nm and a bandwidth of 5nm passes through, and is finally imaged to the focal plane detector 6, and a plurality of high-resolution images 8 with overlapping areas are obtained. The SIFT algorithm (scale-invariant feature transformation algorithm) is used to perform feature point inspection and descriptor extraction on the above-mentioned high-resolution image 8. The descriptors can identify the same feature points after the image transformation, and the corresponding relationship between the feature points is used to generate a point cloud, and high-precision POS data (i.e., position and direction data) is obtained from the point cloud. Since the high-resolution image 8 of the same frame shares posture information with the hyperspectral strip (i.e., the multispectral image 7), the high-precision POS data is used to perform geometric calibration on the multispectral image 7, and the calibrated multispectral image 7 is spliced ​​to finally obtain a hyperspectral image without distortion of the target scene.

[0064] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0065] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A push-broom hyperspectral imaging system for unmanned aerial vehicles, characterized in that: The invention comprises a front lens (1), a line-surface combination aperture (2), a collimating lens (3), a dispersion prism (4), a focusing lens (5) and a focal plane detector (6) which are arranged in sequence, wherein the line-surface combination aperture (2) is provided with a slit (2.1) and a rectangular window (2.2), wherein: The front lens (1) images the target scene onto a rectangular window (2.2), which is collimated by a collimating lens (3) to generate parallel light, which is then passed through a dispersion prism (4) without dispersion, and finally reaches a focal plane detector (6) through a focusing lens (5) to obtain a high-resolution image (8). By comparing the high-resolution image (8) between a current frame and a previous frame, position change information is calculated; At the same time, the front lens (1) also images the target scene onto the slit (2.1), which is collimated by the collimating lens (3) to generate parallel light, which is then dispersed by the dispersion prism (4), and finally reaches the focal plane detector (6) through the focusing lens (5) to obtain a multispectral image (7); The multispectral image (7) of the current frame is calibrated using the posture change information, and the calibrated multispectral images (7) are spliced ​​to obtain a final target scene hyperspectral image.

2. The push-broom hyperspectral imaging system for an unmanned aerial vehicle according to claim 1, characterized in that: A wide-band high-transmittance optical plate (2.3) is provided at the light output of the slit (2.1), and a narrow-band filter (2.4) is provided at the light output of the rectangular window (2.2).

3. The push-broom hyperspectral imaging system for an unmanned aerial vehicle according to claim 2, characterized in that: The wide-band high-transmittance optical plate (2.3) and the narrow-band filter (2.4) have the same thickness, so that the light emitted through the slit (2.1) and the rectangular window (2.2) have the same optical path difference.

4. The push-broom hyperspectral imaging system for an unmanned aerial vehicle according to claim 2, characterized in that: The bandwidth of the narrowband filter (2.4) is less than or equal to the spectral resolution of the hyperspectral imaging system, so as to prevent the image formed by the light emitted from the rectangular window (2.2) on the focal plane detector (6) from being a multi-spectral image.

5. The push-broom hyperspectral imaging system for an unmanned aerial vehicle according to claim 1, characterized in that: The slit (2.1) and the rectangular window (2.2) are arranged in parallel in the length direction, and the distance between the slit (2.1) and the rectangular window (2.2) is in the range of 1 mm to 3 mm.

6. The push-broom hyperspectral imaging system for unmanned aerial vehicles according to claim 1, characterized in that: The distance between the front lens (1) and the line-plane combined aperture (2) is equal to the focal length of the front lens (1), the distance between the line-plane combined aperture (2) and the collimating lens (3) is equal to the focal length of the collimating lens (3), and the distance between the focusing lens (5) and the focal plane detector (6) is equal to the focal length of the focusing lens (5).

7. The push-broom hyperspectral imaging system for an unmanned aerial vehicle according to claim 1, characterized in that: The distance between the collimating lens (3) and the dispersion prism (4) ranges from 1 mm to 2 mm.

8. The push-broom hyperspectral imaging system for an unmanned aerial vehicle according to claim 1, characterized in that: The distance between the dispersion prism (4) and the focusing lens (5) ranges from 1 mm to 2 mm.

9. The push-broom hyperspectral imaging system for an unmanned aerial vehicle according to claim 1, characterized in that: The size of the rectangular window (2.2) is 10 mm*5 mm, and the size of the slit (2.1) is 10 mm*10 μm.

10. The push-broom hyperspectral imaging system for an unmanned aerial vehicle according to claim 1, characterized in that: The front lens (1), the line-surface combined aperture (2), the collimating lens (3), the dispersion prism (4), the focusing lens (5) and the focal plane detector (6) are coaxially arranged.

Images