High-dimensional airborne detection imaging system

Abstract

The application discloses a kind of high-dimensional airborne detection imaging systems, belong to optical imaging field.The system includes protective shell, four black-and-white camera modules, four groups of high-dimensional optical coding devices and two groups of computing units.Four groups of high-dimensional optical coding devices are integrated by four polarizers and four spectral coding filters one-to-one correspondence.The four polarizers are composed of 0 degree linear polarizer, 45 degree linear polarizer, left-handed circular polarizer and right-handed circular polarizer.The system can be integrated with unmanned aerial vehicle, and the images of ground objects collected through the four groups of high-dimensional optical coding devices can be calibrated and configured simultaneously to reduce the errors caused by the different fields of view of the four black-and-white cameras.Finally, the spectral, contour, texture and other dimensional data and image information of the objects can be calculated and reconstructed by combining high-dimensional image decoding algorithm and polarization imaging algorithm.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of optical imaging, and particularly relates to a high-dimensional airborne detection imaging system. BACKGROUND

[0002] The high-dimensional detection imaging system technology combines the advantages of polarization imaging and hyperspectral imaging, and provides an important tool for accurately identifying the composition, surface characteristics and spatial distribution of a target object by acquiring the spectral information and polarization characteristics of the target object at different wavebands. Hyperspectral imaging provides an accurate and reliable means for material identification, classification and monitoring by acquiring fine information of an object at multiple continuous spectral wavebands, such as accurately identifying surface minerals and vegetation types, supporting crop monitoring, food safety detection in agriculture, and quality control in industry, and providing strong support for precise detection and analysis in complex environments such as medical diagnosis, environmental protection and military target identification. At the same time, polarization imaging has significant advantages in dealing with environmental factors such as strong reflection, scattering and haze, can enhance the surface features of an object, remove reflection and haze, and better handle complex lighting environments, and can accurately identify the texture and contour of an object. With the continuous development of technology, polarization hyperspectral imaging will have greater potential in scientific research and engineering applications. SUMMARY

[0003] The present application aims at the deficiencies of the prior art, and provides a high-dimensional airborne detection imaging system.

[0004] The purpose of the present application is achieved by the following technical solution: a high-dimensional airborne detection imaging system, comprising a protective shell, four black-and-white camera modules, four groups of high-dimensional optical encoding devices, two groups of computing units and a bottom connecting adapter;

[0005] The bottom of the protective shell is the camera shooting direction, has four camera openings, and the four groups of high-dimensional optical encoding devices are fixed at the opening positions and randomly correspond to the four black-and-white camera modules fixed inside the shell. The four black-and-white camera modules randomly correspond to the two groups of computing units to acquire the hyperspectral and polarization dimension information of the collected ground objects. The bottom connecting adapter is fixed outside the opening position and is used for integration with a UAV;

[0006] The four groups of high-dimensional optical encoding devices are integrated one-to-one with four different polarization plates and four spectral encoding filters to realize optical encoding of polarization and spectrum;

[0007] The system collects images of the ground objects under the four groups of high-dimensional optical encoding devices, and reconstructs the multi-dimensional data and image information of the objects.

[0008] Further, the four polarizers are respectively a 0-degree linear polarizer, a 45-degree linear polarizer, a left circular polarizer, and a right circular polarizer; and the image of the collected object is obtained through the four polarizers to obtain four hyperspectral images with different polarization states.

[0009] Further, the four spectral encoding filters are formed by alternately stacking titanium dioxide and magnesium fluoride on a glass substrate to form a multilayer film structure, and the polarized image passing through the polarizer to the spectral encoding filter is spectrally encoded, wherein the thickness of each layer is different, and the thickness range is λ / 4-λ, wherein λ is the working wavelength; due to the different thickness of each layer, the spectral encoding filter has different transmittance for different wavelengths of light, thereby realizing the spectral encoding function.

[0010] Further, the four groups of high-dimensional optical encoding devices are used to realize polarization and spectral optical encoding, specifically:

[0011] The initial input image is regarded as a hyperspectral image of HxWxC, H and W are the length and width of the image, and C is the number of spectral channels; after passing through the four polarizers, four hyperspectral images with different polarization states are obtained, which are collectively represented as a high-dimensional image X of HxWxCx4;

[0012] The four hyperspectral images are collected by the black-and-white camera module through the corresponding spectral encoding filter to obtain four gray-scale images, which are the optical encoding results of the selected waveband;

[0013] The black-and-white camera module uploads the collected images to the computing unit, and the computing unit reshapes the high-dimensional image X.

[0014] Further, the computing unit uses a calibration configuration algorithm to process the received images, specifically:

[0015] First, the image is preprocessed to remove noise and highlight edge information, then the contour is extracted through edge detection, the contour that meets the area range is screened, and the polygon approximation is performed on each contour to determine whether it is a quadrilateral, if so, the four vertices are extracted and sorted in the order of top left, top right, bottom right, and bottom left, the coordinates of the vertices are obtained and are persisted;

[0016] In actual processing, the pre-calculated coordinate values are used to uniformly process the images captured by multiple cameras, and the mapping of the extended rectangular region, perspective transformation, and standardized region is used to ensure that the images captured by different cameras can be converted to a unified standard region, and image alignment is realized.

[0017] Further, the computing unit employs a vision transformer to perform spectral reconstruction on the hyperspectral image encoded by the four groups of high-dimensional optical encoders, to obtain a high-dimensional data cube containing spatial and spectral information, i.e., a spectral image, after reconstruction.

[0018] Further, the spectral reconstruction performed by the computing unit is specifically:

[0019] decode the grayscale image of the high-dimensional image X and convert it into a series of flat two-dimensional image blocks X p ; wherein X p has a dimension of N×(P×P×4), (P, P) is the resolution of each image block, and N=(H×W) / (P×P) is the number of image blocks obtained;

[0020] The spectral dimension is expanded by N self-attention mechanism modules, and the spectral curve of each waveband is integrated to obtain a spectral image in the required light wave range.

[0021] The sequence X out is output by the FFN to obtain a high-dimensional data cube containing spatial and spectral information, which has a dimension of N×(P×P×4×C), and the dimension is converted to H×W×C×4.

[0022] Further, in the computing unit, the degree of polarization and the direction of polarization of light of each waveband are read out by calculating Stokes parameters through the linear polarization degree DOLP and the linear polarization angle AOLP formulas.

[0023] First, images of different polarization states under the same wavelength are extracted, and the images collected by the filter corresponding to the 0-degree linear polarizer P0, the 45-degree linear polarizer P45, the left circular polarizer L, the right circular polarizer R, and the polarizer are denoted as I0, I 45 , I left , and I right respectively. The image collected by each polarizer reflects the intensity distribution of light under that polarization state; the Stokes parameters S0, S1, S2, and S3 are calculated, and the formulas are as follows:

[0024] S0=I right +I left

[0025] S0=2I0-S0

[0026] S2=2I 45 -S0

[0027] S3=I right -I left

[0028] Wherein, S0 represents the total intensity of light, S1 represents the intensity difference of light in horizontal and vertical polarization direction, S2 represents the intensity difference of light in 45° and 135° polarization direction, and S3 represents the left and right circular polarization component difference of light.

[0029] Then the linear polarization degree DOLP and linear polarization angle AOLP formula are used to describe the polarization degree and polarization direction of the image at the same wavelength, specifically as follows:

[0030]

[0031]

[0032] Further, in the calculation unit, in order to reconstruct the spectral characteristic curve of the whole image, the following operations are performed:

[0033] The light intensity value S0(x,y,λ) of each pixel point (x,y) in the image at wavelength λ is calculated to obtain a hyperspectral data set of HxWxC, which is a data set under non-polarization state, and the data values of the same pixel point in C images are extracted, and the object is calibrated under the conditions of no light and light;

[0034] No light, i.e. dark field correction, measures the background signal of the instrument without any light to obtain the background value B0(x,y,λ) of the pixel point; Light, i.e. bright field correction, uses a standard white board or reference object with known reflectivity to collect its reflectance spectrum R ref (x,y,λ), and corrects the spectral response difference of the instrument at different wavelengths;

[0035] For the actual spectral value of any pixel point, the following formula is used for correction:

[0036]

[0037] For a pixel point (x,y) in the image, the spectral values at all wavebands are extracted to form a complete spectral curve; then the spectrum of each pixel point is analyzed to draw the spectral characteristics of the entire image.

[0038] Further, the two groups of calculation units use TCP to communicate with each other, one is a master device and the other is a slave device, and when both are ready, the shooting task starts, the master device sends a shooting instruction to the slave device, the slave device starts shooting after receiving the instruction, and simultaneously uses the black and white camera module to obtain images, and all image shooting is completed to send a completion message to the master device; for the master device, after sending the instruction, a compensation time is waited, the compensation time is the average one-way communication time, and then the same shooting process starts, and when all image shooting is completed or exceeds a specified time, the next shooting starts.

[0039] The present application has the following beneficial technical effects:

[0040] 1. The present application can effectively improve the clarity of texture profile recognition through imaging of four polarizers, i.e. 0-degree linear polarization, 45-degree linear polarization, left-handed circular polarization and right-handed circular polarization, and is especially suitable for complex surface and defect detection.

[0041] 2. The present application supports separate disassembly and replacement of the polarizer and the camera module, can adapt to various scene requirements, and reduces maintenance and upgrade costs.

[0042] 3. The bottom connecting adapter of the present application supports various unmanned aerial vehicle mounting interfaces, and the system is light in weight, compact in structure, and can stably operate in a dynamic flight environment.

[0043] 4. Compared with the traditional arrangement and combination of spectral filters and polarizers, the present application can achieve higher image resolution, i.e. the traditional arrangement and combination method requires 4x4=16 optical units, while the present application only needs 4, and the image resolution is increased by 4 times. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0045] Figure 1 A high-dimensional airborne detection imaging system schematic diagram is provided for the embodiments of the present application.

[0046] Figure 2 A high-dimensional airborne detection imaging system top view is provided for the embodiments of the present application.

[0047] Figure 3 A high-dimensional airborne detection imaging system bottom view is provided for the embodiments of the present application.

[0048] Figure 4 A high-dimensional airborne detection imaging system outdoor environment airborne test diagram is provided for the embodiments of the present application.

[0049] Figure 5 A traditional high-dimensional optical coding device array focal plane integrated form design diagram (left) and a four-group high-dimensional optical coding device array focal plane integrated form design diagram (right) provided by the embodiments of the present application.

[0050] Figure 6 A four-group high-dimensional optical coding device design diagram is provided for the embodiments of the present application.

[0051] Figure 7The encoding flow chart of the high-dimensional optical encoding device provided in the embodiment of the present application is shown in the figure.

[0052] Figure 8 The curve measurement chart of the polarizer in the high-dimensional optical encoding device provided in the embodiment of the present application is shown in the figure.

[0053] Figure 9 The structure chart of the spectral encoding filter in the high-dimensional optical encoding device provided in the embodiment of the present application is shown in the figure.

[0054] Figure 10 The spectral curve measurement chart of the spectral encoding filter in the high-dimensional optical encoding device provided in the embodiment of the present application is shown in the figure.

[0055] Figure 11 The decoding flow chart in the spectral reconstruction process provided in the embodiment of the present application is shown in the figure.

[0056] Figure 12 The polarized result generated by the post-processing calculation of the high-dimensional airborne detection imaging system provided in the embodiment of the present application is shown in the figure, from left to right, the original chart, the AOLP algorithm generated chart and the DOLP algorithm generated chart.

[0057] Figure 13 The spectral reconstruction result generated by the post-processing calculation of the high-dimensional airborne detection imaging system provided in the embodiment of the present application is shown in the figure.

[0058] In the figure, 1, left circular polarizer 1; 2, 45-degree linear polarizer; 3, 0-degree linear polarizer; 4, right circular polarizer; 5, polarizer fixing clamp; 6, bottom connecting adapter; 7, protective shell; 8, black and white camera module; 9, calculation unit; 10, power supply. DETAILED DESCRIPTION

[0059] In order to better understand the technical solutions of the present application, the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0060] It should be clear that the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0061] The terms used in the embodiments of the present application are only for the purpose of describing the specific embodiments, and are not intended to limit the present application. The singular forms "a", "an" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0062] As Figures 1-3As shown, the embodiment of the present application provides a high-dimensional airborne detection imaging system, which comprises a protective shell 7, four black-and-white camera modules 8, four groups of high-dimensional optical encoding devices, two groups of computing units 9, a power supply 10 and a bottom connection adapter 6. The bottom of the protective shell 7 is the camera shooting direction, has four camera openings, and four groups of different high-dimensional optical encoding devices are fixed at the opening positions and randomly correspond to the four black-and-white camera modules 8 fixed inside the shell. The four black-and-white camera modules 8 are randomly corresponded to the two groups of computing units 9 to obtain the hyperspectral and polarization dimension information of the collected ground objects. Meanwhile, the bottom connection adapter 6 is fixed outside the opening position, which is used for integration with the unmanned aerial vehicle.

[0063] Further, the four black-and-white camera modules 8 can also be replaced by other components with similar sizes to adapt to more complex imaging requirements. The polarization plate is fixed by the two-sided middle hole polarization plate fixing clamp 5, and the screw hole is opened. The polarization plate is fixed on the protective shell 7 and is independent of each other, and can be individually replaced. The bottom connection adapter 6 supports multiple specifications of unmanned aerial vehicle mounting interfaces, and is convenient for quick disassembly and replacement through modular design.

[0064] Further, as shown, Figure 2 The four black-and-white camera modules 8 all use DMM 37UX290-ML, and the polarization plate is arranged in a cross shape. Every two camera data lines are connected to one computing unit 9, and the two computing units 9 are connected to the power supply 10. Both computing units 9 use Raspberry Pi 4B, and are connected by type A to type C data lines, type C connects the camera, and type A connects the 3.0 port of Raspberry Pi.

[0065] Further, as shown, Figure 3 The power supply 10 is fixed outside the shell, which is convenient for replacement and power supply.

[0066] Further, the two groups of computing units 9 use TCP to communicate with each other, one of which is a master device and the other is a slave device. When both are ready, the shooting task begins, the master device sends a shooting instruction to the slave device, the slave device starts shooting after receiving the instruction, and simultaneously uses the black-and-white camera module to obtain the image. After all the images are shot, a completion message is sent to the master device. For the master device, after sending the instruction, a compensation time is waited, the compensation time is the average one-way communication time, and then the same shooting process is started. When all the images are shot or exceed the specified time, the next shooting is started.

[0067] As shown, Figure 4As shown, four groups of high-dimensional optical encoding devices are integrated one by one with four different polarizers and four self-designed and processed spectral encoding filters, and the optical encoding of polarization and spectrum can be simultaneously realized. The four polarizers are: 0-degree linear polarizer 3, 45-degree linear polarizer 2, left-handed circular polarizer 1 and right-handed circular polarizer 4.

[0068] The present application can reduce the error caused by the different fields of view of the four black-and-white cameras by collecting the images of the ground objects under the four groups of high-dimensional optical encoding devices and through the calibration configuration algorithm. The high-dimensional image decoding algorithm and the polarization imaging algorithm can be combined to calculate and reconstruct the dimensional data and image information of the object, such as spectrum, contour and texture. The overall implementation architecture of the present application includes the acquisition and encoding part and the decoding part.

[0069] In the conventional technology, the polarizer and the spectral encoding filter need to be arranged according to the combination, that is, each two components are combined once. If there are four polarizers and four spectral encoding filters, 4x4=16 optical units are needed. In the present application, each polarizer is one-to-one corresponding to one spectral encoding filter, forming 4 optical units. Figure 5 As shown, in the case of the same optical unit area and the overall area, the number of optical units required is reduced by 4 times, and the area of a single optical unit is increased by 4 times, so that more optical details can be collected, and the resolution of the image is increased by 4 times. As shown, Figure 6 As shown, the four different polarizers and spectral encoding filters can be randomly corresponding, as long as the one-to-one correspondence is met.

[0070] The acquisition and encoding process in the embodiment of the present application is as follows: the initial input image can be regarded as a hyperspectral image with the size of HxWxC, H and W are the length and width of the image, and C is the number of spectral channels. After passing through the four polarizers with different polarization states, four hyperspectral images with different polarization states are obtained, and the overall size is HxWxCx4. The four hyperspectral images are collected by the black-and-white camera module through the corresponding spectral encoding filters to obtain four gray-scale images. The collected gray-scale images are actually the optical encoding results of the selected waveband, and are the input or intermediate step of spectral reconstruction. The overall encoding process is as shown in Figure 7 The specific steps are as follows:

[0071] Step A, the four polarizers are composed of 0-degree linear polarizer P0, 45-degree linear polarizer P45, left-handed circular polarizer L and right-handed circular polarizer R. The image of the object to be collected passes through the four polarizers to obtain four hyperspectral images with different polarization states. The spectral curves of the four polarizers are as shown in Figure 8 The specific steps are as follows:

[0072] Figure 8The black curve Tp in (a) represents the parallel component transmittance of 0-degree linearly polarized light. This curve shows the transmittance of the material to 0-degree linearly polarized light in a certain wavelength range. The red curve Ts represents the perpendicular component transmittance of 0-degree linearly polarized light.

[0073] Figure 8 The black curve Tp in (b) represents the parallel component transmittance of 45-degree linearly polarized light. This curve shows the transmittance of the material to 45-degree linearly polarized light in a certain wavelength range. The red curve Ts represents the perpendicular component transmittance of 45-degree linearly polarized light.

[0074] Figure 8 (c) shows the transmittance of right-handed circularly polarized light into left-handed circularly polarized film LCP and right-handed circularly polarized film RCP.

[0075] Figure 8 (d) shows the transmittance of left-handed circularly polarized light into left-handed circularly polarized film LCP and right-handed circularly polarized film RCP.

[0076] Step B: Four gray-scale images with different polarization states are obtained by the black-and-white camera module through the respective spectral encoding filter.

[0077] As shown in Figure 9 , four self-designed and processed spectral encoding filters are formed by alternately stacking titanium dioxide TiO2 and magnesium fluoride MgF2 on a glass substrate to form a multilayer film structure, which performs spectral encoding on the polarized image passing through the polarizer to the spectral encoding filter (other low refractive index media such as silicon dioxide, aluminum oxide, etc. can also be selected instead of magnesium fluoride).

[0078] wherein the thickness of each layer is different, and the thickness range is about λ / 4-λ, where λ is the working wavelength. Due to the different thickness of each layer, the transmittance of the spectral encoding filter to light of different wavelengths is different, i.e. the transmittance curve of each spectral encoding filter is different, as shown in Figure 10 , thereby realizing spectral selectivity, i.e. spectral encoding function. The four spectral encoding filters are named U1, U2, U3, U4, and have the following related formulas:

[0079]

[0080] wherein X a,i and Y b,i represent the light transmittance of the i-th waveband of the a and b groups of filters, respectively. represent the average light transmittance of the a and b groups of filters, respectively; n represents the total number of wavebands.

[0081] If the spectral selectivity correlation of the four spectral encoding filters is low, that is, there is little overlap between the spectral transmission curves of the filters, each filter corresponds to a different spectral band, and this feature will enable the combined action of different spectral encoding filters to cover a wider wavelength range, and generate a spectral curve for random waveband light for spectral encoding. According to this target spectrum, the microstructure of the four spectral encoding filters is reversely designed, and the random scattering materials and the arrangement conditions of the nanocolumns of the four filters are adjusted, so that the light in each wavelength range can be encoded after passing through the filters, thereby generating a spectral curve and forming spectral information. Therefore, the high-dimensional optical encoding device can simultaneously realize polarization and spectral optical encoding.

[0082] Step C, the spectral image information collected by the four black-and-white cameras is uploaded to two computing units, and the high-dimensional image X is reshaped, and any two of the four black-and-white cameras form a group combined with a computing unit.

[0083] After the computing unit obtains the hyperspectral and polarization images, a calibration configuration algorithm is used, first, the image is preprocessed to remove noise and highlight edge information, then the contour is extracted through edge detection, the contour meeting the area range is screened, and the polygon approximation is performed on each contour to determine whether it is a quadrilateral, if so, the four vertices are extracted and sorted in the order of top left, top right, bottom right and bottom left, and finally the coordinates of the vertices are obtained and persisted; In actual processing, the coordinates calculated in advance are used to uniformly process the images captured by multiple cameras, and the extended rectangular region, perspective transformation and mapping of the standardized region are used to ensure that the images captured by different cameras can be converted to a unified standard region, thereby realizing the consistency and contrast of the images to provide accurate image alignment effect. Thus, the error caused by the different fields of view of the four cameras can be reduced.

[0084] In the decoding process of the embodiment of the application, the decoder architecture uses a network of vision transformer architecture to perform spectral reconstruction on the hyperspectral images encoded by the four high-dimensional optical encoders, to obtain a high-dimensional data cube containing spatial and spectral information after reconstruction, that is, a spectral image. Specifically:

[0085] After calibration, the grayscale image of the high-dimensional image X is decoded and converted into a series of flat two-dimensional image blocks X p ; Wherein X p The dimension is N x (P x P x 4), (P, P) is the resolution of each image block, N = (H x W) / (P x P) is the number of image blocks obtained, also as the effective input sequence length of the vision transformer.

[0086] Through N self-attention mechanism modules, the spectral dimension is expanded, and the spectral curve of each wave band is integrated to obtain the spectral image in the light wave range we need. Finally, the sequence X is output through the FFN out , that is, finally a high-dimensional data cube containing spatial and spectral information is obtained, the cube dimension is N x (P x P x 4 x C), and after dimension conversion, its dimension becomes H x W x C x 4. The decoding process in the spectral reconstruction process is as shown in Figure 11 , which specifically includes the following steps:

[0087] Step a, to read out the polarization degree and polarization direction of light of each wave band, the Stokes parameters are calculated through the linear polarization degree (DOLP) and linear polarization angle (AOLP) formula.

[0088] First, extract the images of different polarization states under the same wavelength. Among them, the images collected by the filter corresponding to the 0-degree linear polarizer P0, 45-degree linear polarizer P45, left-handed circular polarizer L, right-handed circular polarizer R and polarizer are denoted as I0, I 45 , I left , I right , respectively. The image collected by each polarizer reflects the intensity distribution of light under that polarization state. The Stokes parameters S0, S1, S2 and S3 are calculated to completely obtain the polarization state of light, and the calculation formula is as follows:

[0089] S0=I right +I left

[0090] S1=2I0-S0

[0091] S2=2I 45 -S0

[0092] S3=I right -I left

[0093] Among them, S0 represents the total intensity of light (the sum of polarized light and unpolarized light), S1 represents the intensity difference of light in the horizontal and vertical polarization directions, S2 represents the intensity difference of light in the 45° and 135° polarization directions, and S3 represents the left and right circular polarization component difference.

[0094] Then, the linear polarization degree (DOLP) and linear polarization angle (AOLP) formula are used to describe the polarization degree and polarization direction of the image under the same wavelength, specifically:

[0095]

[0096] As Figure 12As shown, the polarization result generated by the late-stage calculation is shown, from left to right, in sequence: a polarization original map, to show the total light intensity distribution of the target object, for overall observation of the brightness characteristics of the object; an AOLP algorithm generated map, to show the polarization direction of light at each pixel point, and since different surface materials or geometric shapes will lead to different polarization directions, the map can also reflect the surface shape, texture and other information of the object; a DOLP algorithm generated map, to show the polarization degree of light at each pixel point, and can highlight the area with significant polarization characteristics.

[0097] Step b, in order to reconstruct the spectral characteristic curve of the whole image, first calculate the light intensity value S0(x,y,λ) of each pixel point (x,y) in the image at wavelength λ, to obtain a hyperspectral data set of HxWxC, which is a data set under non-polarized state, extract the data value of the same pixel point in C images, and calibrate the object under the conditions of no light and light. No light, i.e. dark field correction, measure the background signal of the instrument without any light to obtain the background value B0(x,y,λ) of a pixel point, to remove the interference of system noise on the measurement data. Light, i.e. bright field correction, use a standard white board or reference object with known reflectivity to collect its reflectance spectrum R ref (x,y,λ), to correct the spectral response difference of the instrument at different wavelengths. For the actual spectral value of any pixel point, it can be corrected by the following formula:

[0098]

[0099] Finally, for a pixel point (x,y) in the image, its spectral values at all wavebands can be extracted to form a complete spectral curve; then by analyzing the spectrum of each pixel point, the spectral characteristics of the entire image can be plotted.

[0100] Step c, compare the real spectrum (GT, Ground Truth) and the predicted spectrum (pred, Prediction) using single Gaussian spectrum and double Gaussian spectrum, to evaluate the performance of the spectral reconstruction algorithm, and test the reconstruction ability of the algorithm for single spectrum peak and whether the algorithm can correctly distinguish and restore the characteristics of multiple peaks. Figure 13 Spectrum reconstruction result generated by the late-stage calculation of the embodiment of the present application is shown.

[0101] In addition, the present application can also use the micro-nano process to make four groups of high-dimensional optical encoding devices as an imaging unit, into a form of on-chip array focal plane integration, as a high-dimensional optical encoding chip.

[0102] The above merely describes preferred embodiments of the present application, and the present application is not limited to the above. Any person skilled in the art, without departing from the technical scope of the present application, can make many possible changes and modifications to the technical solutions of the present application, or modify equivalent embodiments with equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application, without departing from the technical scope of the present application, still falls within the protection scope of the technical solutions of the present application.

Claims

1. A high-dimensional airborne detection imaging system, characterized in that, The system comprises a protective shell, four black-and-white camera modules, four sets of high-dimensional optical encoding devices, two sets of computing units and a bottom connecting adapter; The bottom of the protective shell is the direction of camera shooting, has four camera openings, four sets of high-dimensional optical encoding devices are fixed at the opening positions and randomly correspond to the four black-and-white camera modules fixed inside the shell body, the four black-and-white camera modules randomly correspond to the two sets of computing units to obtain the hyperspectral and polarization dimension information of the collected ground objects, the bottom connecting adapter is fixed outside the opening position and is used for integration with the unmanned aerial vehicle; The four sets of high-dimensional optical encoding devices are integrated one-to-one with four different polarization plates and four spectral encoding filters to realize the optical encoding of polarization and spectrum; the four polarization plates are 0-degree linear polarization plate, 45-degree linear polarization plate, left-handed circular polarization plate and right-handed circular polarization plate; the images of the collected objects pass through the four polarization plates to obtain four hyperspectral images with different polarization states; the four spectral encoding filters are formed by alternately stacking titanium dioxide and magnesium fluoride on a glass substrate to form a multilayer film structure, which performs spectral encoding on the polarization images transmitted to the spectral encoding filter through the polarization plate, wherein the thickness of each layer is different, and the thickness range is λ / 4-λ, wherein λ is the working wavelength; due to the different thickness of each layer, the spectral encoding filter has different transmittance for different wavelengths of light, thereby realizing the spectral encoding function; the four sets of high-dimensional optical encoding devices are used to realize the optical encoding of polarization and spectrum, specifically: The initial input image is regarded as a hyperspectral image with HxWxC, H and W are the length and width of the image, and C is the number of spectral channels; after passing through the four polarization plates, four hyperspectral images with different polarization states are obtained, which are collectively represented as a high-dimensional image X with HxWxCx4; The four hyperspectral images pass through the corresponding spectral encoding filters to obtain four gray images collected by the black-and-white camera module, and the gray images are the optical encoding results of the selected waveband; The black-and-white camera module uploads the collected image to the computing unit, and the computing unit remolds the high-dimensional image X; The system collects the images of the ground objects under the four sets of high-dimensional optical encoding devices, and reconstructs the multi-dimensional data and image information of the objects.

2. A high-dimensional airborne detection imaging system according to claim 1, wherein, The computing unit processes the received image by using a calibration configuration algorithm, specifically: First, the image is preprocessed to remove noise and highlight edge information, then the contour is extracted by edge detection, the contour meeting the area range is screened, and the polygon approximation is performed on each contour to determine whether it is a quadrilateral, if so, the four vertices are extracted and sorted in the order of top left, top right, bottom right and bottom left, the coordinates of the vertices are obtained and are persisted; In actual processing, the coordinate values calculated in advance are used to uniformly process the images shot by multiple cameras, and the extended rectangular region, perspective transformation and mapping of the standardized region are used to ensure that the images shot by different cameras can be converted to a unified standard region, thereby realizing image alignment.

3. A high-dimensional airborne detection imaging system according to claim 1, wherein, The computing unit adopts a vision transformer to perform spectral reconstruction on a hyperspectral image coded by four groups of high-dimensional optical encoders, to obtain a high-dimensional data cube containing spatial and spectral information after reconstruction, i.e., a spectral image.

4. A high-dimensional airborne detection imaging system according to claim 3, wherein, The spectral reconstruction performed by the computing unit is specifically: Decoding and converting a gray scale image of a high-dimensional image X into a series of flat two-dimensional image blocks X p ; where X p dimension N x (P x P x 4), (P, P) is the resolution of each image block, N = (H x W) / (P x P) is the number of image blocks obtained; Through N self-attention mechanism modules, the spectral dimension is expanded, and the spectral curve of each waveband is integrated to obtain a spectral image in the required light wave range. Outputting the sequence X by the FFN out , obtaining a high-dimensional data cube containing spatial and spectral information, the cube dimension is N x (P x P x 4 x C), and after dimension conversion, the dimension becomes H x W x C x 4 again.

5. A high-dimensional airborne detection imaging system according to claim 3, wherein, In the computing unit, in order to read the polarization degree and polarization direction of light of each waveband, the Stokes parameters are calculated by using the linear polarization degree DOLP and linear polarization angle AOLP formulas. Firstly, the images of different polarization states under the same wavelength are extracted, and the images collected by the filter corresponding to the 0-degree linear polarizer P0, 45-degree linear polarizer P45, left circular polarizer L, right circular polarizer R and polarizer are denoted as I0, I 45 , I left , I right , respectively. The image collected by each polarizer reflects the intensity distribution of light under the polarization state; the Stokes parameters S0, S1, S2, S3 are calculated, and the formula is as follows: S0 = I right + I left S1 = 2I0-S0 S2 = 2I 45 -S0 S3 = I right - I left Where S0 represents the total intensity of light, S1 represents the intensity difference of light in the horizontal and vertical polarization directions, S2 represents the intensity difference of light in the 45° and 135° polarization directions, and S3 represents the left and right circular polarization component difference of light. Then, the linear polarization degree DOLP and linear polarization angle AOLP formulas are used to describe the polarization degree and polarization direction of the image at the same wavelength, specifically:

6. A high-dimensional airborne detection imaging system according to claim 3, wherein, In the computing unit, in order to reconstruct the spectral characteristic curve of the entire image, the following operations are performed: The light intensity value S0(x,y,λ) of each pixel point (x,y) in the image at the wavelength λ is calculated to obtain a HxWxC hyperspectral data set, which is a data set in the non-polarization state, and the data values of the same pixel point in C images are extracted, and the object is calibrated under the conditions of no light and light. Dark field correction, measure the background signal of the instrument without any light to get the background value B0(x, y, λ) in the pixel; Bright field correction, use a standard white board or reference object with known reflectance to collect its reflectance spectrum R ref (x, y, λ), correct the spectral response difference of the instrument at different wavelengths; For the actual spectral value of any pixel point, the following formula is used for correction: For a pixel point (x,y) in the image, the spectral values of all wavebands are extracted to form a complete spectral curve; then the spectral characteristics of the entire image are drawn by analyzing the spectrum of each pixel point.

7. The high-dimensional airborne detection imaging system of claim 1, wherein, The two groups of computing units use TCP to communicate with each other, one is a master device and the other is a slave device, when both are ready, the shooting task starts, the master device sends a shooting instruction to the slave device, the slave device receives the instruction and starts shooting, and at the same time uses the black and white camera module to obtain images, and sends a completion message to the master device after all image shooting is completed; for the master device, after sending the instruction, wait for a compensation time, the compensation time is the average one-way communication time, and then start the same shooting process, when all image shooting is completed or exceeds the specified time, start the next shooting.

Images