A multi-slit hyperspectral imaging system and method based on unmanned aerial vehicles (UAVs)

Abstract

This invention belongs to the field of spectral imaging technology and relates to a multi-slit hyperspectral imaging system and method based on an unmanned aerial vehicle (UAV), comprising: a front objective lens; a multi-slit assembly including multiple slits for sampling the output image plane of the front objective lens using the multiple slits to obtain multiple sampling strips corresponding to the number of slits; a beam splitting imaging assembly, specifically comprising: a collimating lens group for collimating the multiple sampling strips output by the multi-slit assembly; a dispersive prism group including a first Amish prism and a second Amish prism for dispersing and splitting the multiple sampling strips output by the collimating lens group; a focusing lens group for focusing and imaging the multiple dispersive and split sampling strips output by the dispersive prism group to obtain multiple spectral data; a detector for fusing the multiple spectral data with spatial information to obtain multiple spectral images; and a host computer for fusing the multiple spectral images to obtain a target spectral image.

Description

Technical Field

[0001] This invention relates to the field of spectral imaging technology, and in particular to a multi-slit hyperspectral imaging system and method based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Hyperspectral imaging technology is a combination of imaging and spectral technologies. It is used to detect the two-dimensional geometric space and one-dimensional spectral information of a target, and to acquire continuous, narrow-band image data with high spectral resolution. It has been widely used in agriculture, forestry, oceanography, meteorology, environment and military fields.

[0003] Current hyperspectral imaging technologies are mainly divided into two categories: snapshot and pushbroom. Snapshot hyperspectral imaging technology uses a detector to acquire a complete spectral data cube of the target area within a single frame exposure. The basic imaging idea is to use an optical system to project or separate the three-dimensional data cube of the target spectral area and then tile it onto a two-dimensional detector array. A spectral reconstruction algorithm then restores the three-dimensional spectral data cube from the acquired original two-dimensional data. Because the detector needs to simultaneously collect spatial geometric and spectral information during the imaging process to construct both geometric and spectral images of the target area, it is impossible to simultaneously achieve both spatial and spectral resolution of the acquired pixels. Pushbroom hyperspectral imaging technology uses a pushbroom spectral imaging system to acquire data from the target area. This system consists of a front objective lens, a single-slit assembly, and a planar detector. During the pushbroom process, only one band of hyperspectral data is acquired at each moment. Finally, the complete hyperspectral data of the target area is stitched together based on the hyperspectral data acquired at each moment. Compared to snapshot hyperspectral imaging technology, this imaging method can simultaneously obtain images with high spectral and high spatial resolution. Therefore, the application of pushbroom spectral imaging systems is becoming increasingly widespread.

[0004] Currently, pushbroom hyperspectral imaging systems are widely used in aerospace remote sensing. However, due to the high cost of commonly used aerospace hyperspectral imaging systems and the limitation of aerospace platforms at a certain altitude to perform detailed detection of near-ground areas, new hyperspectral imaging systems based on lightweight UAV platforms are becoming increasingly prevalent. By mounting a pushbroom hyperspectral imaging system on a UAV and controlling its flight path and speed, pushbroom imaging of the target area can be achieved. However, UAVs are susceptible to factors such as airflow and vibration, making it difficult to maintain consistent speed and attitude during the pushbroom process. This can lead to sampling misalignment and missing data, affecting the integrity and accuracy of the hyperspectral image. Furthermore, with the improvement of ground resolution, it is necessary to simultaneously increase the sampling frame rate during pushbroom imaging or reduce the UAV's flight speed to acquire high spatial resolution hyperspectral images. However, increasing the sampling frame rate (i.e., shortening the exposure time) reduces the light throughput of the pushbroom hyperspectral imaging system, thereby reducing the signal-to-noise ratio and decreasing the clarity and contrast of the hyperspectral image. Conversely, reducing the flight speed affects imaging efficiency.

[0005] In summary, existing pushbroom-based spectral imaging systems based on unmanned aerial vehicles (UAVs) suffer from problems such as sampling misalignment, missing samples, and an inability to balance imaging efficiency and imaging quality. Summary of the Invention

[0006] Therefore, the technical problem to be solved by the present invention is to overcome the problems of sampling misalignment, sampling loss, and inability to balance imaging efficiency and imaging quality in the existing pushbroom spectral imaging system based on UAVs.

[0007] To address the aforementioned technical problems, this invention provides a multi-slit hyperspectral imaging system based on an unmanned aerial vehicle (UAV), comprising:

[0008] The front objective lens is used for imaging the input optical path of the area to be imaged;

[0009] A multi-slit assembly, comprising multiple slits, is used to sample the output image plane of the front objective lens using the multiple slits to obtain multiple sampling strips corresponding to the number of slits;

[0010] The spectroscopic imaging component specifically includes:

[0011] The collimating lens group is used to collimate multiple sampling strips output by the multi-slit assembly;

[0012] The dispersive prism assembly includes a first Amish prism and a second Amish prism arranged symmetrically.

[0013] The oblique incident surface of the first Amish prism serves as the incident surface of the dispersive prism group, used to perform dispersive beam splitting on the multiple sampling strips output by the collimating lens group; the oblique incident surface of the second Amish prism serves as the exit surface of the dispersive prism group, used to perform dispersive beam splitting on the multiple sampling strips output by the first Amish prism, and to eliminate aberrations in the multiple sampling strips output by the first Amish prism, outputting multiple dispersive sampling strips;

[0014] A focusing lens group is used to focus and image multiple dispersive and spectral sampled strips output by the dispersive prism group to obtain multiple spectral data corresponding to the multiple dispersive and spectral sampled strips.

[0015] The detector is used to receive the multiple spectral data, acquire spatial information of the area to be imaged, and fuse the multiple spectral data with the spatial information to obtain multiple spectral images;

[0016] The host computer is connected to the detector and is used to fuse the multiple spectral images to obtain the target spectral image of the area to be imaged.

[0017] Preferably, the number of slits in the multi-slit assembly is 3 to 10; and / or

[0018] The width of each slit is 8μm to 25μm; and / or

[0019] The distance between two adjacent slits is 1mm to 1.5mm.

[0020] Preferably, the distance between the first Amish prism and the second Amish prism is 10-20 mm;

[0021] Both the first Amish prism and the second Amish prism are formed by gluing together a right-angle triangular prism and a roof prism.

[0022] Preferably, the apex angles of the right-angle prism and the roof prism are both -15° to 15°;

[0023] The center thickness of the right-angled prism is 8–15 mm;

[0024] The center thickness of the ridge prism is 5-10 mm.

[0025] Preferably, the materials of the right-angle prism and the roof prism include fused silica and calcium fluoride.

[0026] Preferably, the front objective lens is a projection-type double Gaussian objective lens with a focal length of 22.9 mm;

[0027] The multi-slit assembly has 10 slits, each slit is 20 μm wide, and the distance between two adjacent slits is 1 mm.

[0028] The length of the beam splitting imaging component along the optical path propagation direction is 208 mm, and the aperture number of the beam splitting imaging component is 4.5;

[0029] Both the first Amish prism and the second Amish prism are made by bonding a right-angle triangular prism made of H-LAF54 material and a roof prism made of H-ZF88GT material;

[0030] The right-angle prism has an apex angle of 13.5° and a center thickness of 8mm; the roof prism has an apex angle of 5.8° and a center thickness of 5mm.

[0031] The focal length of both the collimating lens group and the focusing lens group is 65mm, and the object space is 0.11.

[0032] Preferably, the host computer includes:

[0033] The feature point recognition module is used to calculate the difference between the gray value of each pixel in each spectral image and the gray value of its neighboring pixels; and to identify pixels whose gray values ​​are all greater than or all less than a first preset threshold as feature points of the spectral image.

[0034] The feature point region construction module is used to obtain a set of pixels in the spectral image whose distance from the feature point is less than a preset distance; and to take every two adjacent pixels in the pixel set as a pixel pair to obtain the pixel pair set of the feature point.

[0035] The feature point descriptor acquisition module is used to obtain the binary feature value of the pixel point pair based on the grayscale values ​​of the first pixel point and the second pixel point in each pixel point pair; and to obtain the binary feature value of the feature point based on the binary feature values ​​of all pixel point pairs in the feature point pixel point pair set.

[0036] The feature point matching module is used to calculate the similarity of each feature point based on the binary feature values ​​of the feature points in each spectral image, so as to superimpose the multiple spectral images to obtain the target spectral image.

[0037] Preferably, the host computer further includes:

[0038] The sampling missing pixel identification module is used to identify pixels with a gray value of 0 in the target spectral image as sampling missing pixels;

[0039] The target pixel acquisition module is used to identify the pixels in each spectral image whose coordinates are the same as those of the sampled missing pixels as target pixels.

[0040] The missing pixel compensation module is used to calculate the average gray value of the target pixels whose gray value is not 0, and use the average gray value as the gray value of the sampled missing pixel.

[0041] This invention also provides a multi-slit hyperspectral imaging method based on unmanned aerial vehicles (UAVs), applied to the aforementioned UAV-based multi-slit hyperspectral imaging system, comprising:

[0042] Image the input optical path of the area to be imaged using a front objective lens;

[0043] The output image plane of the front objective lens is sampled by multiple slits in the multi-slit assembly to obtain multiple sampling strips corresponding to the number of slits;

[0044] The collimating lens group collimates multiple sampling strips output from the multi-slit assembly; the first Amish prism disperses the multiple sampling strips output from the collimating lens group; the second Amish prism group disperses the multiple sampling strips output from the first Amish prism and eliminates aberrations in the multiple sampling strips output from the first Amish prism, outputting multiple dispersed sampling strips; the focusing lens group focuses and images the multiple dispersed sampling strips respectively, obtaining multiple spectral data corresponding to the multiple dispersed sampling strips;

[0045] The detector receives the multiple spectral data, acquires spatial information of the area to be imaged, and fuses the multiple spectral data with the spatial information to obtain multiple spectral images;

[0046] The multiple spectral images are fused to obtain the target spectral image of the region to be imaged.

[0047] Preferably, fusing the multiple spectral images to obtain a target spectral image of the region to be imaged includes:

[0048] Calculate the difference between the gray value of each pixel in each spectral image and the gray value of its neighboring pixels; and select pixels whose gray values ​​are all greater than or all less than a first preset threshold as feature points of the spectral image.

[0049] Obtain the set of pixels in the spectral image whose distance to the feature point is less than a preset distance; take every two adjacent pixels in the set of pixels as a pixel pair to obtain the pixel pair set of the feature point;

[0050] The binary feature value of each pixel pair is obtained based on the grayscale values ​​of the first and second pixels in each pixel pair; the binary feature value of the feature point is obtained based on the binary feature values ​​of all pixel pairs in the pixel pair set of the feature point.

[0051] Based on the binary feature values ​​of feature points in each spectral image, the similarity of each feature point is calculated, and the multiple spectral images are then superimposed to obtain the target spectral image.

[0052] Pixels with a gray value of 0 in the target spectral image are taken as missing sampling pixels;

[0053] In each spectral image, the pixel with the same coordinates as the sampled missing pixel is taken as the target pixel.

[0054] Calculate the average gray value of the target pixels whose gray value is not 0, and use the average gray value as the gray value of the sampled missing pixels.

[0055] The UAV-based multi-slit hyperspectral imaging system provided in this application utilizes multiple slits on a multi-slit assembly to sample different locations within a target area. Multiple sampling strips can be obtained in a single frame, improving both the sampling frame rate and efficiency. Furthermore, the multiple slits design increases the light throughput during sampling, thereby enhancing the signal-to-noise ratio of the imaging system. However, because the multiple sampling strips acquired by the multi-slit assembly may alias, reducing spectral resolution, the spectral images generated based on these strips suffer from poor color reproduction and low contrast. Consequently, the spectral images cannot clearly reflect the characteristics of the target area. Therefore, this application designs a dispersive prism assembly, whose... The system includes a first Amicis prism and a second Amicis prism. Because the refractive indices of the strips collected from different slits are different in the prism material, when these strips are incident from the oblique incident surface of the first Amicis prism, they propagate in different directions according to the law of refraction, resulting in beam dispersion. Simultaneously, due to the symmetrical arrangement of the Amicis prisms, which possess symmetrical optical paths and dispersion characteristics, the second Amicis prism further enhances the dispersion effect of the first Amicis prism and counteracts aberrations present in the strips emitted from the first Amicis prism, effectively separating the strips collected from different slits, preventing aliasing and distortion of strips collected from multiple slits, and improving spectral resolution. This application utilizes multiple slits to simultaneously sample the target area and obtain multiple sampling strips. This not only avoids sampling loss and misalignment problems caused by UAV jitter, improving the light throughput and sampling efficiency of the imaging system, but also improves spectral resolution by dispersing and splitting the strips collected from multiple slits through a dispersive prism group, thus improving imaging efficiency while maintaining image quality. Attached Figure Description

[0056] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein...

[0057] Figure 1 Optical path diagram of the UAV-based multi-slit hyperspectral imaging system provided in this application;

[0058] Figure 2 A schematic diagram of the structure of the UAV-based multi-slit hyperspectral imaging system provided in this application;

[0059] Figure 3 A schematic diagram of a multi-slit assembly provided in this application;

[0060] Figure 4 A schematic diagram of the spectral image output by the detector provided in this application;

[0061] Figure 5 The working principle flowchart of the UAV-based multi-slit hyperspectral imaging system provided in this application;

[0062] Figure 6 A schematic diagram of the target pixel image output by each slit in the UAV-based multi-slit hyperspectral imaging system provided in Example 1;

[0063] Figure 7 The hyperspectral image output by the UAV-based multi-slit hyperspectral imaging system provided in Example 1;

[0064] Explanation of reference numerals in the instruction manual's attached diagrams: 1. Front objective lens; 2. Multi-slit assembly; 3. Beam-splitting imaging assembly; 31. Collimating lens group; 32. Dispersion prism group; 321. First Amish prism; 322. Second Amish prism; a. Right-angle triangular prism; b. Roof prism; 33. Focusing lens group; 4. Detector; 5. Host computer. Detailed Implementation

[0065] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0066] Please see Figure 1 and Figure 2 , Figure 1 The image shown is an optical path diagram of the UAV-based multi-slit hyperspectral imaging system provided in this application. Figure 2 The diagram shown is a schematic of the UAV-based multi-slit hyperspectral imaging system provided in this application. The UAV-based multi-slit hyperspectral imaging system includes a front objective lens 1, a multi-slit assembly 2, a beam splitter assembly 3, a detector 4, and a host computer 5 connected to the detector 4, which are arranged sequentially along the optical path propagation direction.

[0067] The front objective lens 1 is used for imaging the input optical path of the area to be imaged.

[0068] The multi-slit assembly 2 includes multiple slits for sampling the output image plane of the front objective lens 1 using the multiple slits, thereby obtaining multiple sampling strips corresponding to the number of sampling slits.

[0069] The beam-splitting imaging assembly 3 includes a collimating lens group 31, a dispersive prism group 32, and a focusing lens group 33, which are arranged sequentially along the optical path propagation direction.

[0070] The collimating lens group 31 is used to collimate multiple sampling strips output by the multi-slit assembly.

[0071] The dispersive prism group 32 includes a first Amish prism 321 and a second Amish prism 322 arranged symmetrically.

[0072] For example, the first Amish prism 321 and the second Amish prism 322 are symmetrical about the direction perpendicular to the optical axis.

[0073] The oblique incident surface of the first Amish prism 321 serves as the incident surface of the dispersive prism group 32, and is used to perform dispersive beam splitting on the multiple sampling strips output by the straight mirror group 31.

[0074] The oblique incident surface of the second Amish prism 322 serves as the exit surface of the dispersive prism group 32, which is used to perform dispersive beam splitting on the multiple sampling strips output by the first Amish prism 321, and to eliminate the aberrations in the multiple sampling strips output by the first Amish prism 321, thereby outputting multiple dispersive sampling strips.

[0075] The focusing lens group 33 is used to focus and image the multiple dispersive sample strips output by the dispersive prism group 32 to obtain multiple spectral data corresponding to the multiple dispersive sample strips.

[0076] Detector 4 is used to receive multiple spectral data, acquire spatial information of the area to be imaged, and fuse the multiple spectral data with the spatial information to obtain multiple spectral images.

[0077] In a specific example of this application, detector 4 is a two-dimensional surface array detector.

[0078] The host computer 5 is used to fuse multiple spectral images to obtain the target spectral image.

[0079] The UAV-based multi-slit hyperspectral imaging system provided in this application utilizes multiple slits on a multi-slit assembly to sample different locations within a target area. Multiple sampling strips can be obtained in a single frame, improving both the sampling frame rate and efficiency. Furthermore, the multiple slits design increases the light throughput during sampling, thereby enhancing the signal-to-noise ratio of the imaging system. However, because the multiple sampling strips acquired by the multi-slit assembly may alias, reducing spectral resolution, the spectral images generated based on these strips suffer from poor color reproduction and low contrast. Consequently, the spectral images cannot clearly reflect the characteristics of the target area. Therefore, this application designs a dispersive prism assembly, whose... The system includes a first Amicis prism and a second Amicis prism. Because the refractive indices of the strips collected from different slits are different in the prism material, when these strips are incident from the oblique incident surface of the first Amicis prism, they propagate in different directions according to the law of refraction, resulting in beam dispersion. Simultaneously, due to the symmetrical arrangement of the Amicis prisms, which possess symmetrical optical paths and dispersion characteristics, the second Amicis prism further enhances the dispersion effect of the first Amicis prism and counteracts aberrations present in the strips emitted from the first Amicis prism, effectively separating the strips collected from different slits, preventing aliasing and distortion of strips collected from multiple slits, and improving spectral resolution. This application utilizes multiple slits to simultaneously sample the target area and obtain multiple sampling strips. This not only avoids sampling loss and misalignment problems caused by UAV jitter, improving the light throughput and sampling efficiency of the imaging system, but also improves spectral resolution by dispersing and splitting the strips collected from multiple slits through a dispersive prism group, thus improving imaging efficiency while maintaining image quality.

[0080] Furthermore, the number of slits in the multi-slit assembly 2 is 3 to 10, and the number of slits can be 3, 5, 7 or 10.

[0081] Specifically, the number of slits is related to the sampling omission probability. The more slits there are, the lower the sampling omission probability, and the higher the accuracy of the resulting hyperspectral image. For example, if the sampling omission probability of a single-slit component is P, then the sampling omission probability of a multi-slit component with k slits under the same environment is P0. k The signal-to-noise ratio is improved by approximately However, increasing the number of slits inevitably increases the size of the imaging system, making it impossible to meet the requirements for lightweight instruments. Therefore, this application limits the number of slits, thereby reducing the probability of missing samples and improving sampling efficiency while also considering the size of the imaging system.

[0082] Furthermore, the width of each slit is 8μm to 25μm, and the slit width can be 8μm, 10μm, 15μm, 20μm or 25μm.

[0083] Specifically, the slit width affects spectral resolution and spatial resolution. According to the principle of spectral dispersion, the slit width determines the wavelength range of the light it samples. The narrower the slit, the smaller the wavelength range that can pass through, the narrower the spectral bandwidth, and the higher the spectral resolution. Furthermore, when the slit width is too large, light from different positions enters the slit simultaneously and mixes together, reducing the ability to resolve spatial details of the object, thereby reducing spatial resolution. However, when the target area to be imaged is too large, a slit width that is too narrow will affect sampling efficiency. Therefore, this application limits the slit width to improve both spectral resolution and spatial resolution while also taking into account imaging efficiency.

[0084] Furthermore, the spacing between two adjacent slits is 1mm to 1.5mm, and the spacing between adjacent slits can be 1mm, 1.3mm or 1.5mm.

[0085] Specifically, imaging systems are often mounted on drones. The spacing between slits determines whether the drone can accurately repeat sampling of the same ground pixel during push-broom operation, which in turn determines the stitching accuracy of multiple slit sampling strips. Therefore, this application can improve the sampling accuracy of each slit by limiting the spacing between slits, ensuring that multiple sampling strips received by the planar detector have no overlapping parts, thereby improving the stitching accuracy and enabling the hyperspectral image of the target area obtained after stitching to more accurately reflect the feature information of the target area.

[0086] Furthermore, when using the multi-slit assembly 2 for sampling, in order to ensure that the strips obtained from each slit sampling do not overlap, it is necessary to fully separate the multiple sampling strips. The dispersive prism assembly 3 designed in this application utilizes the dispersive characteristics of the Amish prism to prevent the strips collected from multiple slits from overlapping and distortion, thereby improving the spectral resolution.

[0087] Specifically, the distance between the first Amish prism 321 and the second Amish prism 322 is 10-20 mm, and the distance can be 10 mm, 12 mm, 14 mm, 16 mm, 18 mm or 20 mm.

[0088] When the distance between the first amsi prism 321 and the second amsi prism 322 increases, the optical path of the light beam between the first amsi prism 321 and the second amsi prism 322 increases, causing the optical path difference between beams of different wavelengths to change, which in turn affects the distribution and spacing of the interference fringes. The larger the distance, the narrower the interference fringe spacing, the denser the fringes, and the higher the spectral resolution after beam splitting, thus enabling more detailed resolution of light of different wavelengths. However, an excessively large distance will lead to an increase in the size of the imaging system, and the less compact system structure will make it easier for optical elements to shift and deform, resulting in unstable beam splitting effect. Therefore, this application limits the distance between the first amsi prism 321 and the second amsi prism 322 to reduce the system size while ensuring beam splitting effect, and improve the integration and optical stability of the imaging system.

[0089] Furthermore, both the first Amish prism 321 and the second Amish prism 322 are obtained by gluing together a right-angle prism a and a roof prism b.

[0090] Alternatively, the materials for right-angle prism a and roof prism b include fused silica and calcium fluoride.

[0091] Furthermore, the apex angles of both the right-angle prism a and the roof prism b are -15° to 15°, and their apex angles can be -15°, -10°, -5°, 0°, 5°, 10° or 15°.

[0092] Furthermore, the center thickness of the right-angle prism a is 8 to 15 mm, and the center thickness of the right-angle prism a can be 8 mm, 12 mm or 15 mm.

[0093] Furthermore, the center thickness of the ridge prism b is 5-10 mm, and the center thickness of the ridge prism b can be 5 mm, 7 mm or 10 mm.

[0094] For example, such as Figure 3 The diagram shown is a schematic of a multi-slit assembly provided in an embodiment of this application. This multi-slit assembly utilizes 10 slits to perform strip-like discrete sampling of the image plane of the front objective lens 1, with certain gaps between the sampling strips. After the sampling strips are dispersed by the spectroscopic imaging assembly designed in this application, multiple spectral data are output and recorded by a planar detector, resulting in multiple spectral images, such as... Figure 4 The image shown is a spectral image output by a planar detector. As can be seen from the image, the multiple spectral images are independent of each other and do not overlap. Finally, by fusing the multiple spectral images, a hyperspectral image of the target region can be obtained.

[0095] Furthermore, the host computer 5 specifically includes:

[0096] The feature point recognition module is used to calculate the difference between the gray value of each pixel in each spectral image and the gray value of its neighboring pixels; and to identify pixels whose gray values ​​are all greater than or all less than a first preset threshold as feature points of the spectral image.

[0097] The feature point region construction module is used to obtain the set of pixels in the spectral image whose distance from the feature point is less than a preset distance; and to obtain the pixel pair set of the feature point by taking every two adjacent pixels in the pixel set as a pixel pair.

[0098] The feature point descriptor acquisition module is used to obtain the binary feature value of each pixel pair based on the grayscale values ​​of the first and second pixels in each pixel pair; and to obtain the binary feature value of the feature point based on the binary feature values ​​of all pixel pairs in the feature point pixel pair set.

[0099] For example, if the gray value of the first pixel in a pixel pair is greater than the gray value of the second pixel, then the binary feature value of the pixel pair is 1; if the gray value of the first pixel is less than or equal to the gray value of the second pixel, then the binary feature value of the pixel pair is 0. The binary feature value of a feature point is obtained by combining the binary feature values ​​of all pixel pairs in its corresponding pixel pair set.

[0100] The feature point matching module is used to calculate the similarity of each feature point based on the binary feature values ​​of the feature points in each spectral image, so as to superimpose multiple spectral images to obtain the target spectral image.

[0101] Feature point matching based on feature point similarity may result in some missing pixels in the target spectral image obtained by superimposing multiple spectral images, i.e., sampling missing problem. In order to improve the integrity and accuracy of the spectral image and thus more accurately reflect the characteristics of the target region, it is also necessary to identify and supplement the missing pixels in the target spectral image.

[0102] Based on this, in some embodiments of this application, the host computer further includes:

[0103] The sampling missing pixel identification module is used to identify pixels with a gray value of 0 in the target spectral image as sampling missing pixels;

[0104] The target pixel acquisition module is used to identify the pixels in each spectral image whose coordinates are the same as those of the sampled missing pixels as the target pixels.

[0105] The missing pixel compensation module is used to calculate the average gray value of target pixels with a non-zero gray value, and use the average gray value as the gray value of the sampled missing pixels.

[0106] like Figure 5 The diagram shows the working principle flowchart of the UAV-based multi-slit hyperspectral imaging system provided in this application. During a single frame capture, each slit in the multi-slit component samples the target area, obtaining a swath band. As the UAV push-broom process continues, each slit acquires multiple swath bands of the target area. Therefore, after push-brooming, the multiple swath bands sampled by each slit are stitched together to obtain the complete swath corresponding to that slit. Finally, the complete swaths corresponding to all slits are fused to obtain the hyperspectral image of the target area. Because multiple slits sample the same location within the target area during the UAV's ground push-brooming process, repeated sampling of the target area is achieved, reducing the probability of missing samples.

[0107] Specifically, the steps for stitching together multiple strips of slit sampling include: acquiring the position and attitude information of the UAV during the push-broom process from the airborne global positioning system and inertial navigation unit (IMU); calculating the transformation matrix between the strips based on the position and attitude information corresponding to each strip; achieving registration between multiple strips by calculating the transformation matrix; and finally completing the stitching of the strips to obtain the complete strip corresponding to each slit.

[0108] Based on the UAV-based multi-slit hyperspectral imaging system provided in the above embodiments, this application also provides a UAV-based multi-slit hyperspectral imaging method, which specifically includes:

[0109] Image the input optical path of the area to be imaged using a front objective lens;

[0110] The output image plane of the front objective lens is sampled by multiple slits in the multi-slit assembly to obtain multiple sampling strips corresponding to the number of slits;

[0111] The collimating lens group is used to collimate multiple sampling strips output from the multi-slit assembly; the first Amish prism is used to perform dispersive spectral dispersion on the multiple sampling strips output from the collimating lens group; the second Amish prism group is used to perform dispersive spectral dispersion on the multiple sampling strips output from the first Amish prism, and to eliminate aberrations in the multiple sampling strips output from the first Amish prism, outputting multiple dispersive sampling strips; the focusing lens group is used to focus and image the multiple dispersive sampling strips respectively, obtaining multiple spectral data corresponding to the multiple dispersive sampling strips;

[0112] The detector receives multiple spectral data, acquires spatial information of the area to be imaged, and fuses the multiple spectral data with the spatial information to obtain multiple spectral images;

[0113] Multiple spectral images are fused to obtain the target spectral image of the region to be imaged.

[0114] Specifically, in some embodiments of this application, fusing multiple spectral images to obtain a target spectral image of the region to be imaged includes:

[0115] Calculate the difference between the gray value of each pixel in each spectral image and the gray value of its neighboring pixels; and select pixels whose gray values ​​are all greater than or all less than a first preset threshold as feature points of the spectral image.

[0116] Obtain the set of pixels in the spectral image whose distance from the feature point is less than a preset distance; take every two adjacent pixels in the pixel set as a pixel pair to obtain the pixel pair set of the feature point;

[0117] The binary feature value of each pixel pair is obtained based on the grayscale values ​​of the first and second pixels in each pixel pair; the binary feature value of the feature point is obtained based on the binary feature values ​​of all pixel pairs in the pixel pair set of the feature point.

[0118] Based on the binary feature values ​​of feature points in each spectral image, the similarity of each feature point is calculated, and then multiple spectral images are superimposed to obtain the target spectral image;

[0119] Pixels with a gray value of 0 in the target spectral image are taken as missing sampling pixels;

[0120] In each spectral image, the pixel with the same coordinates as the sampled missing pixel is taken as the target pixel.

[0121] Calculate the average gray value of the target pixels whose gray value is not 0, and use the average gray value as the gray value of the sampled missing pixels.

[0122] Existing single-slit hyperspectral imaging systems suffer from strip misalignment and missing data during sampling, leading to severe distortion in the stitched image. Several methods for correcting this distortion have been proposed, including: 1. A shared objective lens optical system for hyperspectral and area array imaging, using low-precision position and attitude data acquired from consumer-grade positioning and inertial navigation devices and an auxiliary area array camera for geometric correction of the hyperspectral image; 2. Utilizing a separate area array camera to provide high-precision attitude data to assist in the geometric correction of the hyperspectral image; 3. Employing scale-invariant feature transformation and random sampling consistency algorithms for geometric correction, while also using additionally acquired RGB images to achieve high-precision correction results. These methods can correct distortion caused by sampling misalignment. For missing sampling areas, existing methods use interpolation algorithms to fill in the missing values. However, interpolation algorithms struggle to recover the spectral data of the missing sampling areas with high fidelity, especially when the UAV is subjected to prolonged disturbances or significant attitude changes, making large-scale sampling loss unrecoverable and resulting in low accuracy and completeness of the hyperspectral image.

[0123] The UAV-based multi-slit hyperspectral imaging method provided in this application acquires spectral information of multiple bands in the imaging area by capturing a single frame, thereby achieving repeated sampling of the target area and reducing the probability of missing samples. Furthermore, when fusing multiple sampled images, feature point identification is first performed on each image, and then the multiple images are superimposed to identify the missing sampling areas. For areas without missing samples, the target gray value of the area can be obtained by calculating the average gray value of all images in that area. For areas with missing samples, the gray value of the missing area is filled by obtaining the average gray value of the images without missing samples in that area. This method reduces the probability of missing samples while designing a new image fusion algorithm, thereby obtaining hyperspectral images with high integrity and accuracy.

[0124] The technical solution of this application will be described in more detail below with reference to several embodiments. However, it should be understood that the following embodiments are only for explaining and illustrating the technical solution and do not limit the scope of this application.

[0125] Example 1

[0126] This embodiment provides a UAV-based multi-slit hyperspectral imaging system, which includes: a front objective lens, a multi-slit assembly, a beam splitting assembly, a detector, and a host computer connected to the detector.

[0127] The front objective lens is a projection double Gaussian objective lens with a focal length of 22.9mm.

[0128] The multi-slit assembly has 10 slits, each slit is 20μm wide, and the spacing between two adjacent slits is 1mm.

[0129] The length of the beam splitting imaging component along the optical path is 208 mm; the aperture number of the beam splitting imaging component is 4.5.

[0130] The spectral imaging assembly includes a collimating lens group, a dispersive prism group, and a focusing lens group; the focal length of both the collimating lens group and the focusing lens group is 65mm, and the object space is 0.11.

[0131] The dispersive prism assembly includes a first Amish prism and a second Amish prism arranged symmetrically; the interval between the first Amish prism and the second Amish prism is 10 mm.

[0132] Both the first and second Amish prisms are made by gluing together a right-angle triangular prism made of H-LAF54 material and a roof prism made of H-ZF88GT material.

[0133] The right-angle prism has an apex angle of 13.5° and a center thickness of 8 mm. The roof prism has an apex angle of 5.8° and a center thickness of 5 mm; light rays undergo a 29° optical path deflection after passing through the first and second Amish prisms.

[0134] Table 1 shows the relevant parameters of the multi-slit component in the UAV-based multi-slit hyperspectral imaging system provided in this embodiment:

[0135] Table 1

[0136] Slit length / mm Slit width / μm Slit spacing / mm Number of slits / slits Field of view / ° 10 20 1 10 47.17

[0137] Table 2 shows the stripe parameters projected onto the ground from the multi-slit assembly shown in Table 1:

[0138] Table 2

[0139] strip width / m Ground sampling interval / m 8.7 0.017

[0140] Table 3 shows the flight parameters for pushbroom photography using a UAV equipped with a multi-slit hyperspectral imaging system:

[0141] Table 3

[0142] <![CDATA[Flight speed / m·s -1 > Flight altitude / m 2 20

[0143] Under the above parameters, the traditional single-slit imaging system is affected by the drone's jitter, which causes the drone's pose to change during the push-broom process. The sampling missing probability of the spectral image obtained by push-broom is 10%, and the image acquisition frame rate is 118pbs.

[0144] In the UAV-based multi-slit hyperspectral imaging system provided in this embodiment, each slit yields 588 sampling strips after a 10m push-broom operation. Based on the actual pose data during the push-broom operation, a projection mapping algorithm is used to stitch together the continuous images corresponding to each slit, such as... Figure 6 As shown in the figure, (1) to (10) are the continuous images corresponding to each slit, where each image contains missing pixels. By superimposing the continuous images corresponding to all slits and compensating for the sampled missing pixels in the target spectral image obtained after fusion, the following is obtained: Figure 7 The hyperspectral image shown.

[0145] By comparing the hyperspectral image obtained in this embodiment with the hyperspectral image obtained by the traditional single-slit imaging system, it was found that the signal-to-noise ratio of the hyperspectral image obtained in this embodiment was increased by 3.2 times compared with that of the hyperspectral image obtained by the traditional single-slit imaging system, and the probability of missing pixels was reduced from 10% to (10%)10.

[0146] Example 2

[0147] This embodiment provides a UAV-based multi-slit hyperspectral imaging system, which includes: a front objective lens, a multi-slit assembly, a beam splitting assembly, a detector, and a host computer connected to the detector.

[0148] The front objective lens is a projection double Gaussian objective lens with a focal length of 22.9mm.

[0149] The multi-slit assembly has three slits, each with a width of 10μm and a spacing of 1.5mm between adjacent slits.

[0150] The length of the beam splitting imaging component along the optical path is 208 mm; the aperture number of the beam splitting imaging component is 4.5.

[0151] The spectral imaging assembly includes a collimating lens group, a dispersive prism group, and a focusing lens group; the focal length of both the collimating lens group and the focusing lens group is 65mm, and the object space is 0.11.

[0152] The dispersive prism assembly includes a first Amish prism and a second Amish prism; the interval between the first Amish prism and the second Amish prism is 15 mm.

[0153] Both the first and second Amish prisms are made by gluing together a right-angle triangular prism made of H-LAF54 material and a roof prism made of H-ZF88GT material.

[0154] The apex angle of a right-angle prism is 0°, and its center thickness is 12.5 mm. The apex angle of a roof prism is 10°, and its center thickness is 7.5 mm.

[0155] Table 4 shows the relevant parameters of the multi-slit component in the UAV-based multi-slit hyperspectral imaging system provided in this embodiment:

[0156] Table 4

[0157] Slit length / mm Slit width / μm Slit spacing / mm Number of slits / slits Field of view / ° 8 10 1.5 3 38.51

[0158] Table 5 shows the strip parameters projected onto the ground from the multi-slit assembly shown in Table 4:

[0159] Table 5

[0160] strip width / m Ground sampling interval / m 8.7 0.017

[0161] Table 6 shows the flight parameters for pushbroom photography using a UAV equipped with a multi-slit hyperspectral imaging system:

[0162] Table 6

[0163] <![CDATA[Flight speed / m·s -1 > Flight altitude / m 1.25 30

[0164] Under the above parameters, the traditional single-slit imaging system is affected by the drone's jitter, which causes the drone's pose to change during the push-broom process. The sampling loss probability of the spectral image obtained by the push-broom is 9%, and the image acquisition frame rate is 96pbs.

[0165] In the UAV-based multi-slit hyperspectral imaging system provided in this embodiment, each slit obtains 385 sampling strips after a push-broom distance of 5m. Based on the actual pose data during push-brooming, a projection mapping algorithm is used to stitch together the continuous images corresponding to each slit. By superimposing the continuous images corresponding to all slits and compensating for the missing sampling pixels in the target spectral image obtained after fusion, a hyperspectral image is obtained.

[0166] By comparing the hyperspectral image obtained in this embodiment with that obtained by a traditional single-slit imaging system, it was found that the signal-to-noise ratio of the hyperspectral image obtained in this embodiment was increased by 1.7 times compared with that of the hyperspectral image obtained by a traditional single-slit imaging system, and the probability of missing pixels was reduced from 9% to (9%)10.

[0167] Example 3

[0168] This embodiment provides a UAV-based multi-slit hyperspectral imaging system, which includes: a front objective lens, a multi-slit assembly, a beam splitting assembly, a detector, and a host computer connected to the detector.

[0169] The front objective lens is a projection double Gaussian objective lens with a focal length of 22.9mm.

[0170] The multi-slit assembly has 6 slits, each slit is 15μm wide, and the spacing between two adjacent slits is 2mm.

[0171] The length of the beam splitting imaging component along the optical path is 208 mm; the aperture number of the beam splitting imaging component is 4.5.

[0172] The spectral imaging assembly includes a collimating lens group, a dispersive prism group, and a focusing lens group; the focal length of both the collimating lens group and the focusing lens group is 65mm, and the object space is 0.11.

[0173] The dispersive prism assembly includes a first Amish prism and a second Amish prism; the interval between the first Amish prism and the second Amish prism is 20 mm.

[0174] Both the first and second Amish prisms are made by gluing together a right-angle triangular prism made of H-LAF54 material and a roof prism made of H-ZF88GT material.

[0175] The apex angle of the right-angle prism is -13.8°, and its center thickness is 15mm. The apex angle of the roof prism is -10°, and its center thickness is 10mm.

[0176] Table 7 shows the relevant parameters of the multi-slit component in the UAV-based multi-slit hyperspectral imaging system provided in this embodiment:

[0177] Table 7

[0178] Slit length / mm Slit width / μm Slit spacing / mm Number of slits / slits Field of view / ° 11 15 2 6 51.31

[0179] Table 8 shows the strip parameters projected onto the ground from the multi-slit assembly shown in Table 7:

[0180] Table 8

[0181] strip width / m Ground sampling interval / m 16.8 0.023

[0182] Table 9 shows the flight parameters for pushbroom photography using a UAV equipped with a multi-slit hyperspectral imaging system:

[0183] Table 9

[0184] <![CDATA[Flight speed / m·s -1 > Flight altitude / m 1.5 35

[0185] Under the above parameters, the traditional single-slit imaging system is affected by the drone's jitter, which causes the drone's pose to change during the push-broom process. The sampling missing probability of the spectral image obtained by push-broom is 15%, and the image acquisition frame rate is 65pbs.

[0186] In the UAV-based multi-slit hyperspectral imaging system provided in this embodiment, each slit obtains 217 sampling strips after a push-broom distance of 5m. Based on the actual pose data during push-brooming, a projection mapping algorithm is used to stitch together the continuous images corresponding to each slit. By superimposing the continuous images corresponding to all slits and compensating for the missing sampling pixels in the target spectral image obtained after fusion, a hyperspectral image is obtained.

[0187] By comparing the hyperspectral image obtained in this embodiment with the hyperspectral image obtained by the traditional single-slit imaging system, it was found that the signal-to-noise ratio of the hyperspectral image obtained in this embodiment was increased by 2.45 times compared with that of the hyperspectral image obtained by the traditional single-slit imaging system, and the probability of missing pixels was reduced from 15% to (15%)10.

[0188] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0189] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0190] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0191] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0192] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A multi-slit hyperspectral imaging system based on an unmanned aerial vehicle (UAV), characterized in that, include: The front objective lens is used for imaging the input optical path of the area to be imaged; A multi-slit assembly, comprising multiple slits, is used to sample the output image plane of the front objective lens using the multiple slits to obtain multiple sampling strips corresponding to the number of slits; The spectroscopic imaging component specifically includes: The collimating lens group is used to collimate multiple sampling strips output by the multi-slit assembly; A dispersive prism group includes a first Amish prism and a second Amish prism symmetrically arranged. The oblique incident surface of the first Amish prism serves as the incident surface of the dispersive prism group, used to disperse and split multiple sampling strips output by the collimating lens group. The oblique incident surface of the second Amish prism serves as the exit surface of the dispersive prism group, used to disperse and split multiple sampling strips output by the first Amish prism, and to eliminate aberrations in the multiple sampling strips output by the first Amish prism, outputting multiple dispersed sampling strips. A focusing lens group is used to focus and image multiple dispersive and spectral sampled strips output by the dispersive prism group to obtain multiple spectral data corresponding to the multiple dispersive and spectral sampled strips. The detector is used to receive the multiple spectral data, acquire spatial information of the area to be imaged, and fuse the multiple spectral data with the spatial information to obtain multiple spectral images; The host computer is connected to the detector and is used to fuse the multiple spectral images to obtain the target spectral image of the area to be imaged.

2. The UAV-based multi-slit hyperspectral imaging system according to claim 1, characterized in that, The number of slits in the multi-slit assembly is 3 to 10; and / or The width of each slit is 8μm to 25μm; and / or The distance between two adjacent slits is 1mm to 1.5mm.

3. The UAV-based multi-slit hyperspectral imaging system according to claim 1, characterized in that, The distance between the first Amish prism and the second Amish prism is 10-20 mm; Both the first Amish prism and the second Amish prism are formed by gluing together a right-angle triangular prism and a roof prism.

4. The UAV-based multi-slit hyperspectral imaging system according to claim 3, characterized in that, The apex angles of the right-angle prism and the roof prism are both -15° to 15°. The center thickness of the right-angled prism is 8–15 mm; The center thickness of the ridge prism is 5-10 mm.

5. The UAV-based multi-slit hyperspectral imaging system according to claim 3, characterized in that, The materials of the right-angle prism and the roof prism include fused silica and calcium fluoride.

6. The UAV-based multi-slit hyperspectral imaging system according to claim 1, characterized in that, The front objective lens is a projection-type double Gaussian objective lens with a focal length of 22.9mm; The multi-slit assembly has 10 slits, each slit is 20 μm wide, and the distance between two adjacent slits is 1 mm. The length of the beam splitting imaging component along the optical path propagation direction is 208 mm, and the aperture number of the beam splitting imaging component is 4.5; Both the first Amish prism and the second Amish prism are made by bonding a right-angle triangular prism made of H-LAF54 material and a roof prism made of H-ZF88GT material; The right-angle prism has an apex angle of 13.5° and a center thickness of 8mm; the roof prism has an apex angle of 5.8° and a center thickness of 5mm. The focal length of both the collimating lens group and the focusing lens group is 65mm, and the object space is 0.

11.

7. The UAV-based multi-slit hyperspectral imaging system according to claim 1, characterized in that, The host computer includes: The feature point recognition module is used to calculate the difference between the gray value of each pixel in each spectral image and the gray value of its neighboring pixels; and to identify pixels whose gray values ​​are all greater than or all less than a first preset threshold as feature points of the spectral image. The feature point region construction module is used to obtain a set of pixels in the spectral image whose distance from the feature point is less than a preset distance; and to take every two adjacent pixels in the pixel set as a pixel pair to obtain the pixel pair set of the feature point. The feature point descriptor acquisition module is used to obtain the binary feature value of the pixel point pair based on the grayscale values ​​of the first pixel point and the second pixel point in each pixel point pair; and to obtain the binary feature value of the feature point based on the binary feature values ​​of all pixel point pairs in the feature point pixel point pair set. The feature point matching module is used to calculate the similarity of each feature point based on the binary feature values ​​of the feature points in each spectral image, so as to superimpose the multiple spectral images to obtain the target spectral image.

8. The UAV-based multi-slit hyperspectral imaging system according to claim 7, characterized in that, The host computer also includes: The sampling missing pixel identification module is used to identify pixels with a gray value of 0 in the target spectral image as sampling missing pixels; The target pixel acquisition module is used to identify the pixels in each spectral image whose coordinates are the same as those of the sampled missing pixels as target pixels. The missing pixel compensation module is used to calculate the average gray value of the target pixels whose gray value is not 0, and use the average gray value as the gray value of the sampled missing pixel.

9. A multi-slit hyperspectral imaging method based on unmanned aerial vehicles (UAVs), characterized in that, The system applied to the UAV-based multi-slit hyperspectral imaging system according to any one of claims 1-8 includes: Image the input optical path of the area to be imaged using a front objective lens; The output image plane of the front objective lens is sampled by multiple slits in the multi-slit assembly to obtain multiple sampling strips corresponding to the number of slits; The collimating lens group collimates multiple sampling strips output from the multi-slit assembly; the first Amish prism disperses the multiple sampling strips output from the collimating lens group; the second Amish prism group disperses the multiple sampling strips output from the first Amish prism and eliminates aberrations in the multiple sampling strips output from the first Amish prism, outputting multiple dispersed sampling strips; the focusing lens group focuses and images the multiple dispersed sampling strips respectively, obtaining multiple spectral data corresponding to the multiple dispersed sampling strips; The detector receives the multiple spectral data, acquires spatial information of the area to be imaged, and fuses the multiple spectral data with the spatial information to obtain multiple spectral images; The multiple spectral images are fused to obtain the target spectral image of the region to be imaged.

10. The multi-slit hyperspectral imaging method based on unmanned aerial vehicles according to claim 9, characterized in that, The multiple spectral images are fused to obtain a target spectral image of the region to be imaged, including: Calculate the difference between the gray value of each pixel in each spectral image and the gray value of its neighboring pixels; and select pixels whose gray values ​​are all greater than or all less than a first preset threshold as feature points of the spectral image. Obtain the set of pixels in the spectral image whose distance to the feature point is less than a preset distance; take every two adjacent pixels in the set of pixels as a pixel pair to obtain the pixel pair set of the feature point; The binary feature value of each pixel pair is obtained based on the grayscale values ​​of the first and second pixels in each pixel pair; the binary feature value of the feature point is obtained based on the binary feature values ​​of all pixel pairs in the pixel pair set of the feature point. Based on the binary feature values ​​of feature points in each spectral image, the similarity of each feature point is calculated, and the multiple spectral images are then superimposed to obtain the target spectral image. Pixels with a gray value of 0 in the target spectral image are taken as missing sampling pixels; In each spectral image, the pixel with the same coordinates as the sampled missing pixel is taken as the target pixel. Calculate the average gray value of the target pixels whose gray value is not 0, and use the average gray value as the gray value of the sampled missing pixels.

Images