High-dimensional airborne detection imaging system

Abstract

The invention discloses a high-dimensional airborne detection imaging system, and belongs to the field of optical imaging. The system comprises a protective shell, four black-and-white camera modules, four groups of high-dimensional optical encoding devices and two groups of calculation units. And the four groups of high-dimensional optical encoding devices are formed by integrating four polaroids and four spectrum encoding optical filters in a one-to-one correspondence manner. The four polaroids are composed of a 0-degree linear polaroid, a 45-degree linear polaroid, a left-hand circular polaroid and a right-hand circular polaroid. The system can be integrated with an unmanned aerial vehicle, images of an earth surface object passing through four groups of high-dimensional optical encoder devices are collected, and errors caused by different view fields of four black-and-white cameras can be reduced through a calibration configuration algorithm. And finally, calculating and reconstructing dimension data such as spectrum, contour, texture and the like and image information of the object by combining a high-dimensional image decoding algorithm and a polarization imaging algorithm.

Description

Technical Field

[0001] The present invention belongs to the field of optical imaging, and particularly relates to a high-dimensional airborne detection imaging system. Background Art

[0002] The technology of high-dimensional detection imaging systems combines the advantages of polarization imaging and hyperspectral imaging. By acquiring the spectral information and polarization characteristics of target objects in different bands, it provides an important tool for accurately identifying the composition, surface characteristics, and spatial distribution of substances. Hyperspectral imaging provides precise and reliable means for material identification, classification, and monitoring by acquiring fine information of an object in multiple continuous spectral bands. For example, it can accurately identify surface minerals and vegetation types through "spectral fingerprints", support crop monitoring in agriculture, food safety detection, and quality control in industry, and also provide strong support for precise detection and analysis in complex environments such as medical diagnosis, environmental protection, and military target identification. At the same time, polarization imaging has significant advantages in dealing with environmental factors such as strong reflections, scattering, and haze. It can enhance the surface features of objects, remove reflections and fog, better handle complex lighting environments, and also accurately identify the texture and contour of objects. With the continuous development of technology, polarization hyperspectral imaging will show greater potential in scientific research and engineering applications. Summary of the Invention

[0003] An object of the present invention is to provide a high-dimensional airborne detection imaging system in view of the deficiencies of the prior art.

[0004] The object of the present invention is achieved by the following technical solutions: A high-dimensional airborne detection imaging system, which includes a protective housing, four black-and-white camera modules, four groups of high-dimensional optical encoding devices, two groups of computing units, and a bottom connection adapter;

[0005] The bottom of the protective housing is the camera shooting direction, and it has four camera openings. Four groups of high-dimensional optical encoding devices are fixed at the opening positions, randomly corresponding to the four black-and-white camera modules fixed inside the housing. The four black-and-white camera modules randomly correspond to the two groups of computing units to obtain the hyperspectral and polarization dimension information of the collected surface objects. The bottom connection adapter is fixed outside the opening position for integration with a drone;

[0006] The four groups of high-dimensional optical encoding devices are integrally formed by corresponding one-to-one a polarizer with four different polarization states and four spectral encoding filters for realizing optical encoding of polarization and spectrum;

[0007] The system acquires images of surface objects under the four groups of high-dimensional optical encoding devices, and calculates and reconstructs the multi-dimensional data and image information of the objects.

[0008] Further, the four polarizers are a 0-degree linear polarizer, a 45-degree linear polarizer, a left-handed circular polarizer, and a right-handed circular polarizer; the image of the object to be collected passes through the four polarizers to obtain four hyperspectral images with different polarization states.

[0009] Further, the four spectral encoding filters are formed by alternately stacking titanium dioxide and magnesium fluoride on a glass substrate to form a multilayer film structure, which performs spectral encoding on the polarized image transmitted through the polarizer. The thickness of each layer is different, and the thickness range is λ / 4 - λ, where λ is the working wavelength; due to the different thicknesses of each layer, the spectral encoding filter has different transmittances for light of different wavelengths, thereby realizing the spectral encoding function.

[0010] Further, the four groups of high-dimensional optical encoding devices are used to realize optical encoding of polarization and spectrum, specifically:

[0011] Regarding the initial input image as a hyperspectral image of H×W×C, where H and W are the length and width of the image, and C is the number of spectral channels; after passing through the four polarizers, four hyperspectral images with different polarization states are obtained, which are collectively represented as a high-dimensional image X of H×W×C×4;

[0012] The four hyperspectral images are respectively collected by a black-and-white camera module through their corresponding spectral encoding filters to obtain four grayscale images, and the grayscale images are the optical encoding results of the selected bands;

[0013] The black-and-white camera module uploads the collected images to the calculation unit, and the calculation unit reshapes the high-dimensional image X.

[0014] Further, the calculation unit processes the received images using a calibration configuration algorithm, specifically:

[0015] First, preprocess the image to remove noise and highlight edge information, then extract the contour through edge detection, screen the contours that meet the area range, perform polygon approximation on each contour, judge whether it is a quadrilateral, if so, extract the four vertices and sort them in the order of upper left, upper right, lower right, and lower left to obtain the coordinates of the vertices and persist them;

[0016] During actual processing, use the pre-calculated coordinate values to uniformly process the images taken by multiple cameras, and utilize the mapping of the extended rectangular area, perspective transformation, and standardized area to ensure that the images taken by different cameras can be converted to a unified standard area to achieve image alignment.

[0017] Further, the computing unit uses a vision transformer to perform spectral reconstruction on the hyperspectral image encoded by four groups of high-dimensional optical encoders, so as to obtain a high-dimensional data cube containing spatial and spectral information after reconstruction, that is, a spectral image.

[0018] Further, the specific process of spectral reconstruction by the computing unit is as follows:

[0019] Decode the grayscale image of the high-dimensional image X and convert it into a series of flat two-dimensional image patches X p ; where X p has a dimension of N×(P×P×4), (P, P) is the resolution of each image patch, and N = (H×W) / (P×P) is the number of obtained image patches;

[0020] Expand the spectral dimension through N self-attention mechanism modules, integrate the spectral curves of each band, and obtain the spectral image within the required light wave range;

[0021] Output the sequence X out through the FFN to obtain a high-dimensional data cube containing spatial and spectral information. The dimension of this cube is N×(P×P×4×C), and after dimension conversion, its dimension becomes H×W×C×4 again.

[0022] Further, in the computing unit, in order to read out the polarization degree and polarization direction of light in each band, the Stokes parameters are calculated through the degree of linear polarization (DOLP) and angle of linear polarization (AOLP) formulas;

[0023] First, extract the images of different polarization states at the same wavelength. The images collected through the 0-degree linear polarizer P0, 45-degree linear polarizer P45, left-handed circular polarizer L, right-handed circular polarizer R, and the corresponding filters of the polarizers are respectively denoted as I 0 , I 45 , I left , I right . The image collected by each polarizer reflects the intensity distribution of light in that polarization state; calculate the Stokes parameters S 0 , S 1 , S 2 , S 3 , and the formulas are as follows:

[0024] S 0 = I right + I left

[0025] S 0 = 2I 0 - S 0

[0026] S 2 = 2I45 -S 0

[0027] S 3 = I right -I left

[0028] where S 0 represents the total intensity of light, S 1 represents the intensity difference of light in the horizontal and vertical polarization directions, S 2 represents the intensity difference of light in the 45° and 135° polarization directions, S 3 represents the difference between the left and right circular polarization components of light;

[0029] Then, the degree of linear polarization DOLP and the angle of linear polarization AOLP formulas are used to describe the polarization degree and polarization direction of the image at the same wavelength, specifically:

[0030]

[0031]

[0032] Furthermore, in the calculation unit, in order to reconstruct the spectral characteristic curve of the entire image, the following operations are performed:

[0033] Calculate the light intensity value S 0 (x, y, λ) of each pixel point (x, y) in the image at wavelength λ, and obtain a hyperspectral data set of H×W×C. This data set is the data set in the unpolarized state. Extract the data values of the same pixel point in C images, and calibrate the object under the conditions of no light and light;

[0034] No light means dark field correction. Measure the background signal of the instrument in the absence of any light to obtain the background value B 0 (x, y, λ); with light means bright field correction. Use a standard whiteboard or reference object with a known reflectivity to collect its reflection spectrum R ref (x, y, λ) to correct the spectral response difference of the instrument at different wavelengths;

[0035] For the actual spectral value of any pixel point, correct it with the following formula:

[0036]

[0037] For a certain pixel point (x, y) in the image, extract its spectral values in all bands to form a complete spectral curve; then, by analyzing the spectrum of each pixel point, plot the spectral characteristics of the entire image.

[0038] Further, the two sets of computing units communicate with each other using TCP. One is the master device and the other is the slave device. When both are ready, the shooting task starts. The master device sends a shooting instruction to the slave device. After receiving the instruction, the slave device starts shooting and simultaneously acquires images using the black-and-white camera module. After all images are taken, a completion message is sent to the master device. For the master device, after sending the instruction, it waits for a compensation time, which is the average one-way communication duration, and then starts the same shooting process. When all images are taken or the specified time is exceeded, the next shooting starts.

[0039] The present invention has the following beneficial technical effects:

[0040] 1. Through imaging with four polarizers of 0-degree linear polarization, 45-degree linear polarization, left-handed circular polarization, and right-handed circular polarization, the present invention can effectively improve the clarity of texture contour recognition, etc., and is particularly suitable for complex surface and defect detection.

[0041] 2. The present invention supports the separate disassembly and replacement of the polarizer and the camera module, can adapt to various scenario requirements, and reduces the maintenance and upgrade costs.

[0042] 3. The adapter connected to the bottom of the present invention supports multiple UAV mounting interfaces. The system is light in weight and compact in structure, and can operate stably in a dynamic flight environment.

[0043] 4. Compared with the traditional method of arranging and combining spectral filters and polarizers, the present invention can achieve higher image resolution. That is, the traditional arrangement and combination method requires 4×4 = 16 optical units, while the present invention only requires 4, and the image resolution is increased by 4 times. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0045] Figure 1 Schematic diagram of the high-dimensional airborne detection imaging system provided by the embodiment of the present invention;

[0046] Figure 2 Top view of the high-dimensional airborne detection imaging system provided by the embodiment of the present invention;

[0047] Figure 3 Bottom view of the high-dimensional airborne detection imaging system provided by the embodiment of the present invention;

[0048] Figure 4Airborne test diagram of the high-dimensional airborne detection imaging system provided by the embodiment of the present invention in an outdoor environment;

[0049] Figure 5 Design diagram of the focal plane integration form of the traditional high-dimensional optical coding device array (left) and the design diagram of the focal plane integration form of the four groups of high-dimensional optical coding device arrays provided by the embodiment of the present invention (right);

[0050] Figure 6 Design diagram of the four groups of high-dimensional optical coding devices provided by the embodiment of the present invention;

[0051] Figure 7 Coding flow chart of the high-dimensional optical coding device provided by the embodiment of the present invention;

[0052] Figure 8 Curve measurement diagram of the polarizer in the high-dimensional optical coding device provided by the embodiment of the present invention;

[0053] Figure 9 Structure diagram of the spectral coding filter in the high-dimensional optical coding device provided by the embodiment of the present invention;

[0054] Figure 10 Spectral curve measurement diagram of the spectral coding filter in the high-dimensional optical coding device provided by the embodiment of the present invention;

[0055] Figure 11 Decoding flow chart during the spectral reconstruction provided by the embodiment of the present invention;

[0056] Figure 12 Example of polarization results generated by post-calculation of the high-dimensional airborne detection imaging system provided by the embodiment of the present invention. From left to right, they are the original image, the image generated by the AOLP algorithm, and the image generated by the DOLP algorithm;

[0057] Figure 13 Spectral reconstruction result generated by post-calculation of the high-dimensional airborne detection imaging system provided by the embodiment of the present invention;

[0058] In the figure: 1. Left-handed circular polarizer 1; 2. 45-degree linear polarizer; 3. 0-degree linear polarizer; 4. Right-handed circular polarizer; 5. Polarizer fixing clip; 6. Bottom connection adapter; 7. Protection housing; 8. Black and white camera module; 9. Calculation unit; 10. Power supply. Detailed implementation manners

[0059] In order to better understand the technical solution of the present application, the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0060] It should be clear that the described embodiments are only a part of the embodiments of this application, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts belong to the scope of protection of this application.

[0061] The terms used in the embodiments of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms "a", "the" and "said" used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0062] As Figure 1-3 shown, the embodiments of the present invention provide a high-dimensional airborne detection imaging system, including a protective housing 7, four black-and-white camera modules 8, four groups of high-dimensional optical coding devices, two groups of computing units 9, a power supply 10, and a bottom connection adapter 6; the bottom of the protective housing 7 is the camera shooting direction, with four camera openings, and four groups of different high-dimensional optical coding devices are fixed at the opening positions, randomly corresponding to the four black-and-white camera modules 8 fixed inside the housing, and the four black-and-white camera modules 8 are also randomly corresponding to the two groups of computing units 9 to obtain the hyperspectral and polarization dimension information of the collected surface objects. At the same time, the bottom connection adapter 6 is fixed outside the opening position for integration with the drone.

[0063] Furthermore, the four black-and-white camera modules 8 can also be replaced with other components of similar size to adapt to more complex imaging requirements. Use a polarization sheet fixing clip 5 with a hole dug in the middle on both sides to clamp the polarization sheet, and drill screw holes to fix it on the protective housing 7, and they are independent of each other and can be disassembled and replaced separately. The bottom connection adapter 6 supports a variety of specifications of drone mounting interfaces and is convenient for quick disassembly, assembly and replacement through modular design.

[0064] Furthermore, as Figure 2 shown, all four black-and-white camera modules 8 use DMM 37UX290-ML and are arranged in a cross shape like the polarization sheet; the data lines of every two cameras are connected to one computing unit 9, the two computing units 9 are connected to the power supply 10, and both two computing units 9 use Raspberry Pi 4B and are connected with a type A to type C data cable, with type C connected to the camera and type A connected to the 3.0 port of the Raspberry Pi.

[0065] Furthermore, as Figure 3 shown, the power supply 10 is fixed outside the housing, which is convenient for replacement and power replenishment.

[0066] Furthermore, the two sets of computing units 9 communicate with each other using TCP. One is the master device and the other is the slave device. When both are ready, the shooting task starts. The master device sends a shooting instruction to the slave device. The slave device receives the instruction and starts shooting, and at the same time uses the black and white camera module to acquire images. After all the images are taken, a completion message is sent to the master device. For the master device, after sending the instruction, it waits for a compensation time, which is the average one-way communication duration, and then starts the same shooting process. When all the images are taken or the specified time is exceeded, the next shooting starts.

[0067] As Figure 4 shown, the four sets of high-dimensional optical encoding devices are integrally formed by polarizers with four different polarization states and four spectrally encoded filters designed and processed independently, corresponding one by one, and can simultaneously achieve optical encoding of polarization and spectrum. The four polarizers are: linear polarizer 3 at 0 degrees, linear polarizer 2 at 45 degrees, left-handed circular polarizer 1, and right-handed circular polarizer 4.

[0068] By collecting the images of surface objects through the four sets of high-dimensional optical encoding devices and simultaneously passing through the calibration configuration algorithm, the present invention can reduce the error caused by the different fields of view of the four black and white cameras; combined with the high-dimensional image decoding algorithm and the polarization imaging algorithm, it can respectively calculate and reconstruct the spectral, contour, texture and other dimensional data and image information of the object. The overall implementation architecture of the present invention includes an acquisition and encoding part and a decoding part.

[0069] In the traditional technology, the polarizer and the spectrally encoded filter need to be arranged in combinations, that is, every two components are combined once. If there are four polarizers and four spectrally encoded filters, 4×4 = 16 sets of optical units are required. In the present invention, each polarizer only corresponds to one spectrally encoded filter one by one, forming 4 sets of optical units. As Figure 5 shown, when the area of the optical unit is the same as the overall area, since the number of required optical units is reduced by 4 times, the area of a single optical unit is increased by 4 times, so that more optical details can be collected, and the resolution of the image is increased by 4 times. Among them, as Figure 6 shown, the polarizers with four different polarization states and the spectrally encoded filters can correspond randomly, as long as the one-to-one correspondence relationship is satisfied.

[0070] In the embodiments of the present invention, the acquisition and encoding process is specifically as follows: The initial input image can be regarded as a hyperspectral image of H×W×C, where H and W are the length and width of the image, and C is the number of spectral channels. After passing through polarizers with four polarization states, four hyperspectral images with different polarization states are obtained respectively, which can be represented as a high-dimensional image X with a size of H×W×C×4 as a whole. The four hyperspectral images are respectively collected by the black-and-white camera module through their corresponding spectral encoding filters to obtain four grayscale images. The collected grayscale images are actually the optical encoding results of the selected bands and are the input or intermediate steps of spectral reconstruction. The overall encoding process is as Figure 7 shown, and specifically includes the following steps:

[0071] Step A: The four polarizers are composed of a 0-degree linear polarizer P0, a 45-degree linear polarizer P45, a left-handed circular polarizer L, and a right-handed circular polarizer R; the image of the object to be collected passes through the four polarizers to obtain four hyperspectral images with different polarization states. The spectral curves of the four polarizers are as Figure 8 shown, specifically:

[0072] Figure 8 The black curve Tp in (a): represents the transmittance of the parallel component of the 0-degree linearly polarized light. This curve shows the transmittance of the material to the 0-degree linearly polarized light in a specific wavelength range. The red curve Ts: represents the transmittance of the vertical component of the 0-degree linearly polarized light.

[0073] Figure 8 The black curve Tp in (b): represents the transmittance of the parallel component of the 45-degree linearly polarized light. This curve shows the transmittance of the material to the 45-degree linearly polarized light in a specific wavelength range. The red curve Ts: represents the transmittance of the vertical component of the 45-degree linearly polarized light.

[0074] Figure 8 In (c), it shows the transmittance of the right-handed circular polarized light entering the left-handed circular polarizer LCP and the right-handed circular polarizer RCP.

[0075] Figure 8 In (d), it shows the transmittance of the left-handed circular polarized light entering the left-handed circular polarizer LCP and the right-handed circular polarizer RCP.

[0076] Step B: The hyperspectral images with different polarization states are respectively collected by the black-and-white camera module through their corresponding spectral encoding filters to obtain four grayscale images.

[0077] As Figure 9 shown, the four self-designed and processed spectral encoding filters are composed of titanium dioxide TiO 2 and magnesium fluoride MgF 2An alternating stack is formed on a glass substrate to form a multilayer film structure, and spectral encoding is performed on the polarized image transmitted through a polarizer to a spectral encoding filter (other low-refractive-index media such as silica and alumina can also be selected to replace magnesium fluoride).

[0078] Among them, the thickness of each layer is different, and the thickness range is about λ / 4 - λ, where λ is the working wavelength. Since the thickness of each layer is different, the spectral encoding filter has different light transmittance for different wavelengths, that is, the transmittance curve of each spectral encoding filter is different, as Figure 10 shown, so as to achieve spectral selectivity, that is, the spectral encoding function. The four spectral encoding filters are named U1, U2, U3, and U4 respectively, and there are the following correlation formulas:

[0079]

[0080] Among them, X a,i , Y b,i respectively represent the light transmittance of the a and b groups of filters in the i-th band. respectively represent the average light transmittance of the a and b groups of filters; n represents the total number of bands.

[0081] If the spectral selectivity correlation of the four spectral encoding filters is low, that is, the overlapping part between their spectral transmittance curves is very small, and each filter corresponds to a different spectral band, this characteristic will enable different spectral encoding filters to work together to cover a wider wavelength range and generate a spectral curve for spectral encoding of randomly banded light. According to this target spectrum, the microstructures of the four spectral encoding filters of the system are reversely designed, and the random scattering materials and nano-column arrangement conditions of these four filters are adjusted so that the light in each wavelength range can be specifically encoded after passing through these filters, thereby generating a spectral curve and forming spectral information. Thus, the high-dimensional optical encoding device can simultaneously achieve optical encoding of polarization and spectrum.

[0082] Step C: The four black-and-white cameras upload the collected spectral image information to two computing units to reshape the high-dimensional image X. Any two of the four black-and-white cameras are grouped with one computing unit.

[0083] After the computing unit obtains the hyperspectral and polarization images, it adopts the calibration configuration algorithm. First, it preprocesses the images to remove noise and highlight edge information, then extracts the contours through edge detection, filters the contours that meet the area range, and performs polygon approximation on each contour to determine whether it is a quadrilateral. If so, it extracts the four vertices and sorts them in the order of upper left, upper right, lower right, and lower left. Finally, it obtains the coordinates of the vertices and persists them. During the actual processing, it uses the pre-calculated coordinate values to uniformly process the images taken by multiple cameras, and uses the mapping of the extended rectangular area, perspective transformation, and standardized area to ensure that the images taken by different cameras can be converted to a unified standard area, so as to achieve the consistency and comparability of the images and provide an accurate image alignment effect. Thus, the error caused by the different fields of view of the four cameras can be reduced.

[0084] During the decoding process of the embodiment of the present invention, the decoder architecture adopts a network of the vision transformer architecture to perform spectral reconstruction on the hyperspectral images encoded by four groups of high-dimensional optical encoders, so as to obtain a high-dimensional data cube containing spatial and spectral information after reconstruction, that is, a spectral image. Specifically:

[0085] After calibration, the grayscale image of the high-dimensional image X is decoded and converted into a series of flat two-dimensional image blocks X p ; where X p has a dimension of N×(P×P×4), (P,P) is the resolution of each image block, and N=(H×W) / (P×P) is the number of obtained image blocks, which is also used as the effective input sequence length of the vision transformer.

[0086] Through N self-attention mechanism modules, the spectral dimension is expanded, and the spectral curves of each band are integrated to obtain the spectral image within the required light wave range. Finally, the output sequence X out is obtained, that is, finally a high-dimensional data cube containing spatial and spectral information is obtained. The dimension of this cube is N×(P×P×4×C), and after the dimension transformation, its dimension becomes H×W×C×4 again. The decoding process during spectral reconstruction is as Figure 11 shown, specifically including the following steps:

[0087] Step a: To read out the polarization degree and polarization direction of the light in each band, the Stokes parameters are calculated through the degree of linear polarization (DOLP) and angle of linear polarization (AOLP) formulas.

[0088] First, the images of different polarization states at the same wavelength are extracted. Among them, the images collected through the 0-degree linear polarizer P0, 45-degree linear polarizer P45, left-handed circular polarizer L, right-handed circular polarizer R, and the corresponding filters of the polarizers are respectively denoted as I 0 、I45 , I left , I right , the image collected by each polarizer reflects the intensity distribution of light in that polarization state. Calculate the Stokes parameters S 0 , S 1 , S 2 , S 3 , to fully obtain the polarization state of light, the calculation formulas are as follows:

[0089] S 0 = I right + I left

[0090] S 1 = 2I 0 - S 0

[0091] S 2 = 2I 45 - S 0

[0092] S 3 = I right - I left

[0093] Among them, S 0 represents the total intensity of light (the sum of polarized light and unpolarized light), S 1 represents the intensity difference of light in the horizontal and vertical polarization directions, S 2 represents the intensity difference of light in the 45° and 135° polarization directions, S 3 represents the difference between the left and right circular polarization components of light.

[0094] Then use the degree of linear polarization (DOLP) and angle of linear polarization (AOLP) formulas to describe the polarization degree and polarization direction of the image at the same wavelength, specifically:

[0095]

[0096] As Figure 12 shown, the examples of polarization results generated by later calculations are, from left to right: the original polarization image, to show the total light intensity distribution of the target object for overall observation of the brightness characteristics of the object; the image generated by the AOLP algorithm, to show the polarization direction of light at each pixel point, and at the same time, because different surface materials or geometric shapes will result in different polarization directions, so this image can also reflect information such as the surface shape and texture of the object; the image generated by the DOLP algorithm, to show the polarization degree of light at each pixel point, and can highlight the regions with significant polarization characteristics.

[0097] Step b: To reconstruct the spectral characteristic curve of the entire image, first calculate the light intensity value S of each pixel point (x, y) in the image at wavelength λ0 (x, y, λ) to obtain a hyperspectral dataset of H×W×C. This dataset is under non-polarized state. Extract the data values of the same pixel point in C images, and calibrate the object under lightless and lighted conditions. Lightless means dark field correction, which measures the background signal of the instrument without any illumination to obtain the background value B in a pixel point. 0 (x, y, λ) to remove the interference of system noise on the measured data. Lighted means bright field correction, which uses a standard whiteboard or reference object with known reflectivity to collect its reflection spectrum R. ref (x, y, λ) to correct the spectral response differences of the instrument at different wavelengths. For the actual spectral value of any pixel point, it can be corrected by the following formula:

[0098]

[0099] Finally, for a pixel point (x, y) in the image, the spectral values at all bands can be extracted to form a complete spectral curve; then by analyzing the spectra of each pixel point, the spectral characteristics of the entire image can be plotted.

[0100] Step c: Compare the real spectrum (GT, Ground Truth) and the predicted spectrum (pred, Prediction) using single Gaussian spectrum and double Gaussian spectrum to evaluate the performance of the spectral reconstruction algorithm, and at the same time test the reconstruction ability of the algorithm for a single spectral peak and whether the algorithm can correctly distinguish and restore the characteristics of multiple peaks. Figure 13 This is an example of the spectral reconstruction result generated in the later stage of the embodiment of the present invention.

[0101] In addition, the present invention can also use micro-nano technology to take four groups of high-dimensional optical encoding devices as an imaging unit and fabricate them in the form of an on-chip array focal plane integration as a high-dimensional optical encoding chip.

[0102] The above are only the preferred embodiments of the present invention. Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make many possible changes and modifications to the technical solution of the present invention, or modify it into an equivalent embodiment with equivalent changes, without departing from the scope of the technical solution of the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the protection of the technical solution of the present invention.

Claims

1. A high-dimensional airborne detection imaging system, characterized in that: It includes a protective shell, four black-and-white camera modules, four sets of high-dimensional optical encoding devices, two sets of computing units and bottom connection adapters; The bottom of the protective shell is the camera shooting direction, with four camera openings, four sets of high-dimensional optical encoding devices are fixed at the opening positions, and randomly correspond to the four black-and-white camera modules fixed inside the shell, and the four black-and-white camera modules are randomly corresponding to the two sets of computing units to obtain the high-spectral and polarization dimension information of the collected surface objects, and the bottom connection adapter is fixed outside the opening position for integration with the drone; The four groups of high-dimensional optical encoding devices are integrated with four polarizers with different polarization states and four spectral encoding filters in a one-to-one correspondence, and are used to realize optical encoding of polarization and spectrum; The system collects images of surface objects through four groups of high-dimensional optical encoding devices, and reconstructs multi-dimensional data and image information of the objects by calculation.

2. A high-dimensional airborne detection imaging system according to claim 1, characterized in that: The four polarizers are respectively a 0-degree linear polarizer, a 45-degree linear polarizer, a left-handed circular polarizer, and a right-handed circular polarizer; after the image of the collected object passes through the four polarizers, four hyperspectral images with different polarization states are obtained.

3. A high-dimensional airborne detection imaging system according to claim 1, characterized in that: The four spectral coding filters are formed by alternately stacking titanium dioxide and magnesium fluoride on a glass substrate to form a multilayer film structure, which spectrally encodes the polarized image transmitted to the spectral coding filter through the polarizer, wherein the thickness of each layer is different, and the thickness range is λ / 4-λ, wherein λ is the working wavelength; due to the different thickness of each layer, the spectral coding filter has different transmittance to light of different wavelengths, thereby realizing the spectral coding function.

4. A high-dimensional airborne detection imaging system according to claim 1, characterized in that: The four groups of high-dimensional optical encoding devices are used to realize optical encoding of polarization and spectrum, specifically: The initial input image is regarded as a H×W×C hyperspectral image, where H and W are the length and width of the image, and C is the number of spectral channels. After passing through four polarizers, four hyperspectral images with different polarization states are obtained, which are represented as a H×W×C×4 high-dimensional image X. The four hyperspectral images are collected by the black and white camera module through the corresponding spectral coding filters to obtain four grayscale images, and the grayscale images are the optical coding results of the selected bands; The black and white camera module uploads the collected image to the computing unit, and the computing unit reshapes the high-dimensional image X.

5. A high-dimensional airborne detection imaging system according to claim 4, characterized in that: The calculation unit processes the received image using a calibration configuration algorithm, specifically: First, the image is preprocessed to remove noise and highlight edge information. Then, the contour is extracted through edge detection. The contours that meet the area range are screened, and each contour is approximated by polygons to determine whether it is a quadrilateral. If so, the four vertices are extracted and sorted in the order of upper left, upper right, lower right, and lower left. The coordinates of the vertices are obtained and persisted. During actual processing, the images taken by multiple cameras are uniformly processed using the coordinate values ​​calculated in advance, and the extended rectangular area, perspective transformation, and standardized area mapping are utilized to ensure that the images taken by different cameras can be converted to a unified standard area to achieve image alignment.

6. A high-dimensional airborne detection imaging system according to claim 4, characterized in that: The computing unit uses a vision transformer to perform spectral reconstruction on the hyperspectral image encoded by four groups of high-dimensional optical encoders to obtain a high-dimensional data cube containing spatial and spectral information after reconstruction, namely, a spectral image.

7. A high-dimensional airborne detection imaging system according to claim 6, characterized in that: The calculation unit performs spectrum reconstruction specifically as follows: Decode the grayscale image of the high-dimensional image X and convert it into a series of flat two-dimensional image blocks X p ; where X p The dimension is N×(P×P×4), (P,P) is the resolution of each image block, and N=(H×W) / (P×P) is the number of image blocks obtained; Through N self-attention mechanism modules, the spectral dimension is expanded, and the spectral curve of each band is integrated to obtain the spectral image within the required light wave range; Output sequence X through FFN out , we get a high-dimensional data cube containing spatial and spectral information, the dimension of which is N×(P×P×4×C). After the transformation, its dimension becomes H×W×C×4 again.

8. A high-dimensional airborne detection imaging system according to claim 6, characterized in that: In the calculation unit, in order to read out the polarization degree and polarization direction of light in each band, the Stokes parameters are calculated by the linear polarization degree DOLP and linear polarization angle AOLP formulas; First, images of different polarization states at the same wavelength are extracted. The images collected through the 0-degree linear polarizer P0, the 45-degree linear polarizer P45, the left-hand circular polarizer L, the right-hand circular polarizer R and the filters corresponding to the polarizers are recorded as I0, I 45 ,I left ,I right , the image collected by each polarizer reflects the intensity distribution of light under the polarization state; calculate the Stokes parameters S0, S1, S2, S3, the formula is as follows: S0=I right +I left S1=2I0-S0 <h2 style=";text-align:left;direction:ltr">S2=2I<h2 style=";text-align:left;direction:ltr"> 45 <h2 style=";text-align:left;direction:ltr"> -S0 S3=I right -I left Among them, S0 represents the total intensity of light, S1 represents the intensity difference between the horizontal and vertical polarization directions of light, S2 represents the intensity difference between the 45° and 135° polarization directions of light, and S3 represents the difference between the left and right circular polarization components of light; Then the degree of linear polarization DOLP and angle of linear polarization AOLP formulas are used to describe the polarization degree and polarization direction of the image at the same wavelength, specifically:

9. A high-dimensional airborne detection imaging system according to claim 6, characterized in that: In the calculation unit, in order to reconstruct the spectral characteristic curve of the entire image, the following operations are performed: Calculate the light intensity value S0(x,y,λ) of each pixel point (x,y) in the image at wavelength λ to obtain a H×W×C hyperspectral data set, which is a data set in a non-polarized state. Extract the data value of the same pixel point in C images and calibrate the object under no light and light conditions; Dark field calibration is used to measure the background signal of the instrument without any light to obtain the background value B0(x, y, λ) in the pixel point; bright field calibration is used to collect the reflection spectrum R of a standard white plate or reference object with known reflectivity. ref (x,y,λ), corrects for differences in the instrument's spectral response at different wavelengths; For the actual spectral value of any pixel, use the following formula to correct it: For a certain pixel point (x, y) in the image, its spectral values ​​in all bands are extracted to form a complete spectral curve; then the spectral characteristics of the entire image are plotted by analyzing the spectrum of each pixel point.

10. The high-dimensional airborne detection imaging system according to claim 1, characterized in that: The two groups of computing units communicate with each other using TCP, one of which is the master device and the other is the slave device. When the two are ready at the same time, the shooting task begins. The master device sends a shooting instruction to the slave device, and the slave device starts shooting after receiving the instruction, and uses the black and white camera module to acquire images at the same time. After all images are shot, a completion message is sent to the master device. For the master device, after sending the instruction, a compensation time is waited, and the compensation time is the average one-way communication time, and then the same shooting process is started. When all images are shot, or the specified time is exceeded, the next shooting begins.

Images