A pushbroom hyperspectral imaging system for unmanned aerial vehicles

Abstract

This invention relates to a pushbroom hyperspectral imaging system for unmanned aerial vehicles (UAVs). A line-plane combined aperture has a slit and a rectangular window. A front-facing lens images the target scene into the rectangular window. The light is collimated by a collimating lens to generate parallel light, then passes through a dispersive prism without dispersion, and finally reaches a focal plane detector through a focusing lens, obtaining a high-resolution image. By comparing the high-resolution images between adjacent frames, pose change information is obtained. The front-facing lens also images the target scene into the slit, collimated by a collimating lens to generate parallel light, then passes through a dispersive prism to cause dispersion, and finally reaches a focal plane detector through a focusing lens, obtaining a multispectral image. The pose change information is used to calibrate the multispectral image of the current frame, and the calibrated images are then stitched together to obtain the final hyperspectral image of the target scene. This invention has a simple and easy-to-use structure, requiring no additional system to assist in image stitching.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral imaging technology, and in particular to a pushbroom hyperspectral imaging system for unmanned aerial vehicles (UAVs). Background Technology

[0002] In the 1980s, with the development of Earth observation applications, imaging spectroscopy technology emerged. It is an emerging field that integrates space imaging technology and spectral imaging technology, greatly expanding the scope of applications in aerospace remote sensing, agricultural and forestry resource detection, mineral resource and geological exploration, military reconnaissance, biomedicine and other fields.

[0003] With the increasing maturity of drone technology, the new pushbroom hyperspectral imaging technology based on a lightweight drone platform has gained favor among remote sensing professionals. It has made the originally difficult, time-consuming, and costly remote sensing data acquisition work simple, fast, and low-cost.

[0004] Today, UAV-borne push-broom hyperspectral imaging systems have demonstrated strong application potential and market value in various fields such as agriculture, water environment monitoring, forestry management, disaster relief, and aerospace, thanks to their efficient and accurate data acquisition capabilities.

[0005] Pushbroom hyperspectral imaging technology can only acquire hyperspectral data of one band of ground at a time. After being pushed by an UAV, a complete data cube of the target area is obtained. This type of system can obtain data with high spatial and hyperspectral resolution and is widely used on satellite platforms.

[0006] The primary challenge in extending this technology to UAV platforms is the stability of the UAVs. Lightweight UAVs operating in the troposphere are susceptible to airflow and vibration, making it difficult to maintain consistent speed and attitude during flight. This leads to misalignment or missing ground sampling during the pushbroom process of the hyperspectral imaging system. Such unstable sampling results in severe distortion of the geometric image after strip stitching. Current geometric calibration methods for pushbroom hyperspectral images mainly focus on post-processing of data based on POS data or auxiliary area array cameras. For example, the team led by Xue Qingsheng at Ocean University of China proposed an optical system that uses a shared objective lens for hyperspectral imaging and area array imaging. This system employs low-precision POS (Position and Orientation System) data acquired by consumer-grade positioning and inertial navigation devices (GPS / IMU) and an auxiliary area array camera to perform geometric calibration on the pushbroom hyperspectral image. First, low-precision POS data is used for preliminary calibration of the pushbroom strips. Then, high-precision relative POS information is extracted from the synchronously acquired inter-frame area array images for geometric registration of the hyperspectral image. Teams led by Zhao Chunjiang and Yang Guijun at the Beijing Agricultural Information Technology Research Center, Jurado's team at the University of Jaén in Spain, Li Haiwei's team at Xi'an Jiaotong University, and VVShipko from the Special Instruments Science and Technology Center of the Russian Academy of Sciences, among others, utilized independent area array cameras to provide high-precision POS data to assist in the geometric calibration of hyperspectral images. Zhang Hao's team at the Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, used only low-precision POS information and employed Scale Invariant Feature Transform (SIFT) and Random Sample Consensus (RANSAC) algorithms to perform geometric calibration on the scanned images. He also pointed out that using additional RGB images would yield even higher-precision calibration results.

[0007] In summary, the above solutions mainly rely on additional systems to assist image stitching, such as using a POS system or an area scan camera to acquire pose change information for image correction. This approach cannot guarantee synchronization and increases weight and cost. Therefore, how to achieve interference-resistant acquisition of hyperspectral images while ensuring synchronization as much as possible and reducing weight and cost is a pressing problem that UAV pushbroom hyperspectral imaging systems need to solve. Summary of the Invention

[0008] Therefore, the technical problem to be solved by the present invention is to overcome the problem that the pushbroom hyperspectral imaging system in the prior art requires the addition of an additional system (such as a POS system or an area array camera) to provide pose change information to assist in image stitching correction, which in turn cannot guarantee synchronization and also increases weight.

[0009] To address the aforementioned technical problems, this invention provides a pushbroom-type hyperspectral imaging system for unmanned aerial vehicles (UAVs), comprising a front-facing lens, a line-plane combined aperture, a collimating lens, a dispersive prism, a focusing lens, and a focal plane detector arranged sequentially. The line-plane combined aperture has a slit and a rectangular window.

[0010] The front-facing lens images the target scene onto a rectangular window, which is then collimated by a collimating lens to generate parallel light. The light then passes through a dispersive prism without dispersion and finally reaches the focal plane detector through a focusing lens to obtain a high-resolution image. By comparing the high-resolution images between the current frame and the previous frame, the pose change information is calculated.

[0011] Simultaneously, the front-facing lens also images the target scene to the slit, which is then collimated by the collimating lens to generate parallel light. The light is then dispersed by the dispersive prism and finally reaches the focal plane detector through the focusing lens to obtain a multispectral image.

[0012] The pose change information is used to calibrate the multispectral image of the current frame, and the calibrated multispectral images are then stitched together to obtain the final hyperspectral image of the target scene.

[0013] In one embodiment of the present invention, a wide-band high-transmittance optical plate is provided at the light emission point of the slit, and a narrow-band filter is provided at the light emission point of the rectangular window.

[0014] In one embodiment of the present invention, the wideband high transmittance optical plate and the narrowband filter have the same thickness, so that the light emitted through the slit and the rectangular window has the same optical path difference.

[0015] In one embodiment of the present invention, the bandwidth of the narrowband filter is less than or equal to the spectral resolution of the hyperspectral imaging system, so as to prevent the image formed on the focal plane detector by the light emitted from the rectangular window from being a multispectral image.

[0016] In one embodiment of the present invention, the slit and the rectangular window are arranged parallel to each other in the length direction, and the distance between the slit and the rectangular window is in the range of 1mm-3mm.

[0017] In one embodiment of the present invention, the distance between the front lens and the line-plane combination aperture is equal to the focal length of the front lens, the distance between the line-plane combination aperture and the collimating lens is equal to the focal length of the collimating lens, and the distance between the focusing lens and the focal plane detector is equal to the focal length of the focusing lens.

[0018] In one embodiment of the present invention, the distance between the collimating lens and the dispersive prism is in the range of 1mm-2mm.

[0019] In one embodiment of the present invention, the distance between the dispersive prism and the focusing lens is in the range of 1mm-2mm.

[0020] In one embodiment of the present invention, the rectangular window has a size of 10mm*5mm, and the slit has a size of 10mm*10μm.

[0021] In one embodiment of the present invention, the front-facing lens, the line-plane combination aperture, the collimating lens, the dispersive prism, the focusing lens, and the focal plane detector are arranged coaxially.

[0022] The technical solution of the present invention has the following advantages compared with the prior art:

[0023] The pushbroom hyperspectral imaging system for UAVs described in this invention constructs a line-plane combined aperture, which has a slit and a rectangular window. Hyperspectral information is obtained from the multispectral image formed by the object surface passing through the slit, and pose change information is obtained from the high-resolution image formed by the object surface passing through the rectangular window. It does not require an additional area array camera to provide images and obtain pose change information from them, nor does it require a separate GPS / IMU to provide POS data. This invention is simple and effective.

[0024] This invention effectively reduces the weight of the hyperspectral imaging system, ensures synchronization, and has a simple overall structure, making it suitable for large-scale deployment. Attached Figure Description

[0025] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0026] Figure 1 This is a schematic diagram of the pushbroom hyperspectral imaging system for UAVs in an embodiment of the present invention;

[0027] Figure 2 This is a first-view schematic diagram of a line-plane combined aperture in an embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of the second perspective of the line-plane combined aperture in an embodiment of the present invention;

[0029] Figure 4 This is a side view of the line-plane combined aperture in an embodiment of the present invention.

[0030] Explanation of reference numerals in the accompanying drawings: 1. Front lens; 2. Linear-plane combined aperture; 2.1. Slit; 2.2. Rectangular window; 2.3. Wideband high-transmittance optical plate; 2.4. Narrowband filter; 3. Collimating lens; 4. Dispersion prism; 5. Focusing lens; 6. Focal plane detector; 7. Multispectral image; 8. High-resolution image; 9. Computer. Detailed Implementation

[0031] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0032] Example 1

[0033] Reference Figure 1 As shown, the present invention relates to a pushbroom hyperspectral imaging system for unmanned aerial vehicles, comprising a front lens 1, a line-plane combined aperture 2, a collimating lens 3, a dispersive prism 4, a focusing lens 5, and a focal plane detector 6 arranged in sequence, and these optical hardware components need to be placed coaxially.

[0034] Furthermore, the focal plane detector 6 in this embodiment is also connected to the computer 9, which is used to receive data from the focal plane detector 6 and calculate the final hyperspectral image of the target scene.

[0035] Please see Figure 2 The linear-plane combined aperture 2 has a slit 2.1 and a rectangular window 2.2. The front lens 1 images the target scene into the rectangular window 2.2, which is then collimated by the collimating lens 3 to produce parallel light. After passing through the dispersive prism 4, no dispersive light occurs. Finally, the light reaches the focal plane detector 6 through the focusing lens 5, resulting in a high-resolution image 8. By comparing the high-resolution images 8 between the current frame and the previous frame, the pose change information is calculated. Simultaneously, the front lens 1 also images the target scene into the slit 2.1, which is collimated by the collimating lens 3 to produce parallel light. After passing through the dispersive prism 4, the light is dispersed and finally reaches the focal plane detector 6 through the focusing lens 5, resulting in a multispectral image 7. The multispectral image 7 of the current frame is calibrated using the pose change information. The calibrated multispectral images 7 are then stitched together to obtain the final hyperspectral image of the target scene. It should be noted that the optical plate 2.3 corresponding to the slit 2.1 does not filter light, so the light passing through the optical plate 2.3 will be dispersed; while the narrowband filter 2.4 corresponding to the rectangular window 2.2 filters light, so the light passing through the narrowband filter 2.4 will not be dispersed. The optical plate 2.3 and the narrowband filter 2.4 will be described in detail below.

[0036] Please see Figure 3 and Figure 4A wide-band high-transmittance optical plate 2.3 is provided at the light exit point of the slit 2.1. This wide-band high-transmittance optical plate 2.3 is used in conjunction with subsequent optical elements to provide a multispectral image 7. A narrow-band filter 2.4 is provided at the light exit point of the rectangular window 2.2. This narrow-band filter 2.4 is used in conjunction with subsequent optical elements to provide a high-resolution image 8. Pose change information is obtained through the high-resolution images 8 between adjacent frames. It should be noted that, according to... Figure 2 and Figure 3 It is not difficult to see that both the slit 2.1 and the rectangular window 2.2 are hollowed out on the linear-surface combined aperture 2. In this embodiment, the side of the linear-surface combined aperture 2 facing the front lens 1 is the incident light side, and the side of the linear-surface combined aperture 2 facing the collimating lens 3 is the outgoing light side. That is, the wide-band high-transmittance optical plate 2.3 is set on the outgoing light side of the slit 2.1, and the narrow-band filter 2.4 is set on the outgoing light side of the rectangular window 2.2.

[0037] The following is a detailed description of this embodiment:

[0038] The relevant specifications of the pushbroom hyperspectral imaging system for UAVs in this embodiment are as follows:

[0039] The spectral range of the hyperspectral imaging system is 400–900 nm.

[0040] The spectral resolution of the hyperspectral imaging system is an average of 5 nm.

[0041] The narrowband filter 2.4 has a center wavelength of 550nm and a bandwidth of 5nm;

[0042] Narrowband filter 2.4 thickness: 0.5mm-1mm, preferably 0.5mm;

[0043] Narrowband filter 2.4 dimensions: 10mm * 0.5mm;

[0044] The thickness of the wideband high transmittance optical plate 2.3 is 0.5mm-1mm, and it needs to be consistent with the thickness of the narrowband filter 2.4, preferably 0.5mm;

[0045] Wideband high transmittance optical flat plate 2.3 Dimensions: 10mm*5mm;

[0046] The distance between the rectangular window 2.2 and the slit 2.1 is 1mm-3mm, preferably 2mm;

[0047] Rectangular window 2.2 dimensions: 10mm * 5mm;

[0048] The dimensions of the line-plane combined aperture 2 are: 10mm*12mm*0.5mm;

[0049] Slit 2.1 dimensions: 10mm * 10μm;

[0050] Focal plane detector 6-pixel size: 10μm × 10μm;

[0051] Focal plane detector 6 scale: 2560×1280 pixels;

[0052] Front camera 1 focal length: 40mm-50mm, preferably 40mm;

[0053] Collimating lens 3 focal length: 40mm-50mm, preferably 40mm;

[0054] Focal length of focusing lens 5: 40mm-50mm, preferably 40mm;

[0055] Dispersion prism 4: Direct-view Amish prism;

[0056] The object-side half-field of view of the hyperspectral imaging system is 5.71°.

[0057] Furthermore, in this embodiment, the front-facing lens 1 is an image-side telecentric lens.

[0058] Furthermore, in this embodiment, the length of slit 2.1 is 10 mm, thus the object-side field of view of the hyperspectral imaging system is calculated to be 13.25°-14.25°, preferably 14.25°. Simultaneously, the light passing through the front lens 1 must cover both slit 2.1 and rectangular window 2.2. The diameter of the smallest circumcircle of slit 2.1 and rectangular window 2.2 is 12.21 mm, resulting in a calculated object-side field of view of 33°-34°, preferably 34°. It should be noted that in this embodiment, the object-side field of view of the hyperspectral imaging system must satisfy the condition that the object image covers slit 2.1, and the object-side field of view of the front lens 1 must satisfy the condition that the object image covers both slit 2.1 and rectangular window 2.2.

[0059] Furthermore, a preset distance (1mm-3mm) is left between the slit 2.1 and the rectangular window 2.2. This preset distance is used to ensure that the high-resolution image 8 formed by the object surface through the rectangular window 2.2 and the multispectral image 7 formed by the object surface through the slit 2.1 will not overlap.

[0060] refer to Figure 1 Regarding the relevant specifications of the pushbroom hyperspectral imaging system for UAVs provided in this embodiment, the target scene needs to be imaged through the front lens 1 at the line-plane junction aperture 2. Therefore, the distance between the front lens 1 and the line-plane junction aperture 2 is the focal length of the front lens 1. Similarly, the distance between the line-plane junction aperture 2 and the collimating lens 3 is the focal length of the collimating lens 3.

[0061] After being collimated by the collimating lens 3, the light becomes parallel. Therefore, the distance between the collimating lens 3 and the dispersive prism 4 should be as small as possible to reduce the system size. In this embodiment, it is set to 1mm-2mm, preferably 1mm. Similarly, after being dispersed by the dispersive prism 4, the light remains parallel. Therefore, the distance between the dispersive prism 4 and the focusing lens 5 should also be as small as possible. In this embodiment, it is set to 1mm-2mm, preferably 1mm. The dispersed light needs to pass through the focusing lens 5 to form a multispectral image 7 on the focal plane detector 6 to provide hyperspectral information. The undispersed light needs to pass through the focusing lens 5 to form a high-resolution image 8 on the focal plane detector 6 to provide pose change information. Therefore, the distance between the focusing lens 5 and the focal plane detector 6 is the focal length of the focusing lens 5.

[0062] refer to Figure 2 and Figure 3 In this embodiment, the linear-plane combined aperture 2 has a slit 2.1 and a rectangular window 2.2. A narrowband filter 2.4 needs to be added behind the rectangular window 2.2 to prevent the image formed on the focal plane detector 6 by the light emitted from the rectangular window 2.2 from being a multispectral image. Similarly, a wide-band high-transmittance optical plate 2.3 with the same thickness as the narrowband filter 2.4 needs to be added behind the slit 2.1 to avoid a difference in optical path difference between the light emitted from the slit 2.1 and the rectangular window 2.2. Specifically, in this embodiment, to prevent the image formed on the focal plane detector 6 by the light emitted from the rectangular window 2.2 from being a multispectral image, the bandwidth (5nm) of the narrowband filter 2.4 is not greater than the spectral resolution (5nm) of the hyperspectral imaging system. The spectral resolution of the hyperspectral imaging system is determined by the pixel size of the slit 2.1 and the focal plane detector 6.

[0063] Furthermore, the light passing through the rectangular window 2.2 is filtered by a narrow-band filter 2.4, allowing only light with a center wavelength of 550nm and a bandwidth of 5nm to pass through. This light is then imaged onto the focal plane detector 6, resulting in multiple high-resolution images 8 with overlapping regions. The SIFT algorithm (Scale Invariant Feature Transform) is used to perform feature point verification and descriptor extraction on the high-resolution images 8. The descriptors identify identical feature points after image transformation, and the correspondence between these feature points is used to generate a point cloud. High-precision POS data (i.e., position and orientation data) is obtained from the point cloud. Since the high-resolution image 8 and the hyperspectral stripe (i.e., multispectral image 7) share pose information, the high-precision POS data is used to perform geometric calibration on the multispectral image 7. The calibrated multispectral images 7 are then stitched together to obtain a distortion-free hyperspectral image of the target scene.

[0064] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0065] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A pushbroom hyperspectral imaging system for unmanned aerial vehicles (UAVs), characterized in that: The system includes a front-facing lens (1), a line-plane combined aperture (2), a collimating lens (3), a dispersive prism (4), a focusing lens (5), and a focal plane detector (6), arranged sequentially. The line-plane combined aperture (2) has a slit (2.1) and a rectangular window (2.2). The front-facing lens (1) images the target scene onto a rectangular window (2.2), which is then collimated by the collimating lens (3) to generate parallel light. After passing through the dispersive prism (4), no dispersive light is generated. Finally, the light reaches the focal plane detector (6) through the focusing lens (5) to obtain a high-resolution image (8). By comparing the high-resolution image (8) between the current frame and the previous frame, the pose change information is calculated. Meanwhile, the front lens (1) also images the target scene to the slit (2.1), which is collimated by the collimating lens (3) to generate parallel light, and then dispersed by the dispersive prism (4). Finally, it reaches the focal plane detector (6) through the focusing lens (5) to obtain a multispectral image (7). The pose change information is used to calibrate the multispectral image (7) of the current frame, and the calibrated multispectral image (7) is stitched together to obtain the final target scene hyperspectral image.

2. The pushbroom hyperspectral imaging system for unmanned aerial vehicles according to claim 1, characterized in that: A wide-band high-transmittance optical plate (2.3) is provided at the light emission point of the slit (2.1), and a narrow-band filter (2.4) is provided at the light emission point of the rectangular window (2.2).

3. The pushbroom hyperspectral imaging system for unmanned aerial vehicles according to claim 2, characterized in that: The wideband high transmittance optical plate (2.3) and the narrowband filter (2.4) have the same thickness so that the light emitted through the slit (2.1) and the rectangular window (2.2) has the same optical path difference.

4. The pushbroom hyperspectral imaging system for UAVs according to claim 2, characterized in that: The bandwidth of the narrowband filter (2.4) is less than or equal to the spectral resolution of the hyperspectral imaging system, so as to avoid the image formed on the focal plane detector (6) by the light emitted from the rectangular window (2.2) being a multispectral image.

5. The pushbroom hyperspectral imaging system for unmanned aerial vehicles according to claim 1, characterized in that: The slit (2.1) and the rectangular window (2.2) are arranged parallel to each other in the length direction, and the distance between the slit (2.1) and the rectangular window (2.2) is in the range of 1mm-3mm.

6. The pushbroom hyperspectral imaging system for unmanned aerial vehicles according to claim 1, characterized in that: The distance between the front lens (1) and the line-plane combination aperture (2) is equal to the focal length of the front lens (1), the distance between the line-plane combination aperture (2) and the collimating lens (3) is equal to the focal length of the collimating lens (3), and the distance between the focusing lens (5) and the focal plane detector (6) is equal to the focal length of the focusing lens (5).

7. The pushbroom hyperspectral imaging system for unmanned aerial vehicles according to claim 1, characterized in that: The distance between the collimating lens (3) and the dispersive prism (4) is in the range of 1mm-2mm.

8. The pushbroom hyperspectral imaging system for unmanned aerial vehicles according to claim 1, characterized in that: The distance between the dispersive prism (4) and the focusing lens (5) is in the range of 1mm-2mm.

9. The pushbroom hyperspectral imaging system for unmanned aerial vehicles according to claim 1, characterized in that: The rectangular window (2.2) has a size of 10mm*5mm, and the slit (2.1) has a size of 10mm*10μm.

10. The pushbroom hyperspectral imaging system for unmanned aerial vehicles according to claim 1, characterized in that: The front lens (1), the line-plane combined aperture (2), the collimating lens (3), the dispersive prism (4), the focusing lens (5), and the focal plane detector (6) are coaxially arranged.

Images