Spectral imaging system based on adaptive super-resolution

Abstract

The spectral imaging system comprises a coaxial common-aperture objective lens, a collimating lens, a dichroic light splitting element, a coding template, a hyperspectral imaging assembly, a piezoelectric ceramic micro-scanning super-resolution device and a multispectral detection assembly. The coaxial common-aperture objective lens converges the target light to the collimating lens, and the target light is shaped into parallel light beams by the collimating lens; the dichroic light splitting element splits the parallel light beams and forms a visible light / near-infrared light branch light path and an intermediate infrared light beam branch light path; the coding template and the hyperspectral imaging assembly are arranged on the visible light / near-infrared light branch light path; and the piezoelectric ceramic micro-scanning super-resolution device and the multispectral detection assembly are arranged on the mid-infrared light beam branch light path. The system utilizes a self-feedback mechanism to carry out self-adaptive infrared space super-resolution imaging and visible spectrum super-resolution imaging, so that the detection capability can be ensured, the data volume is reduced, and the real-time processing of data is ensured.

Description

Technical Field

[0001] The present invention relates to the field of spectral imaging technology, and particularly relates to a spectral imaging system based on adaptive super-resolution. Background Art

[0002] Currently, infrared image super-resolution technology mainly focuses on how to use deep learning or multi-frame fusion technology to improve the ability to restore image details. Li Yan et al. in the paper "Infrared Image Super-Resolution Reconstruction Method Based on Deep Learning" adopted a multi-layer convolutional neural network and a feature fusion strategy to perform end-to-end training on a large number of low-resolution infrared images, so as to learn to restore image details and texture information. However, since this method relies on a large dataset and offline training, its model has a weak adaptive adjustment ability in real-time scenarios and is difficult to meet the high timeliness requirements in dynamic environments. The patent application "Infrared Image Super-Resolution Reconstruction Method, Device, Equipment and Storage Medium and Process" with the application number CN2024113011084 adopts a strategy based on multi-frame image fusion and motion compensation, and improves the spatial resolution of the image through precise image registration and offline reconstruction. However, this solution also focuses on the offline processing of pre-acquired images and lacks a dynamic adjustment function for real-time target changes. In the field of hyperspectral super-resolution, Xu et al. in the paper "Hyperspectral Image Super-Resolution Using a Generative Adversarial Network" published in the IEEE Access 2019 journal proposed an offline super-resolution reconstruction method based on a generative adversarial network to restore hyperspectral image details through a deep network. However, this method does not combine spectral encoding with the real-time acquisition process and is difficult to simultaneously balance spatial and spectral resolutions in dynamic scenarios.

[0003] Most of the existing methods generate higher-resolution and more spectral-dimensional data in a machine learning manner, and their authenticity is doubtful, and there is a contradiction between extremely large data volume and limited real-time processing ability.

[0004] In view of this, the present invention proposes a spectral imaging system based on adaptive super-resolution. In contrast, this system realizes the self-feedback acquisition of high-spatial-resolution and high-spectral-dimensional data of the target of interest through intelligent automatic feedback regulation. This system adopts a coarse detection mode to reduce the data volume. After detecting a suspicious target, it uses a self-feedback mechanism to perform adaptive infrared spatial super-resolution imaging and spectral super-resolution imaging in the visible spectral band, so as to not only ensure the detection ability but also reduce the data volume and ensure the real-time processing of the data. Summary of the Invention

[0005] The object of the present invention is to propose a spectral imaging system based on adaptive super-resolution; by using a self-feedback mechanism, adaptive infrared spatial super-resolution imaging and spectral super-resolution imaging in the visible spectral band are carried out, so as to not only ensure the detection ability, but also reduce the amount of data and ensure the real-time processing of data.

[0006] To achieve the above object, the technical solution of the present invention is as follows: A spectral imaging system based on adaptive super-resolution includes a coaxial common-aperture objective lens, a collimator, a dichroic beam splitter element, a coding template, a hyperspectral imaging component, a piezoelectric ceramic micro-scanning super-resolution device, and a multispectral detection component; The optical axes of the coaxial common-aperture objective lens and the collimator coincide; the coaxial common-aperture objective lens converges the target light to the collimator, and the collimator reshapes it into a parallel light beam; the dichroic beam splitter element splits the parallel light beam into a visible / near-infrared light beam and a mid-infrared light beam according to a preset splitting ratio, forming a visible / near-infrared light branch optical path and a mid-infrared light beam branch optical path; The coding template and the hyperspectral imaging component are sequentially arranged on the visible / near-infrared light branch optical path along the propagation direction of the visible / near-infrared light beam; the coding template modulates the visible / near-infrared light beam at the pixel level through a micromirror array to improve the spectral signal-to-noise ratio and resolution; the hyperspectral imaging component includes a dispersive element and a detector, and after the coded visible / near-infrared light beam is separated into spectra of each band by the dispersive element, a coded hyperspectral image is generated on the detector; The piezoelectric ceramic micro-scanning super-resolution device and the multispectral detection component are sequentially arranged on the mid-infrared light beam branch optical path along the propagation direction of the mid-infrared light beam; the multispectral detection component includes a multi-channel beam splitter device and a multi-channel detector, and after the multi-channel beam splitter device splits the beam bandwidth passing through the piezoelectric ceramic micro-scanning super-resolution device into multiple narrow-band channels, it enters the multi-channel detector to complete the synchronous acquisition of a multispectral infrared image.

[0007] Preferably, the preset splitting ratio is 50:50.

[0008] Preferably, the dispersive element includes a prism or a grating.

[0009] Preferably, the multi-channel beam splitter device includes a multi-channel band-pass filter or a multi-prism beam splitter.

[0010] Preferably, the multi-channel beam splitter device and the multi-channel detector respectively adopt a four-channel beam splitter device and a four-channel detector.

[0011] Preferably, it further includes an adjustable optical bracket, and the coaxial common-aperture objective lens, the collimator, the dichroic beam splitter element, the coding template, the hyperspectral imaging component, the piezoelectric ceramic micro-scanning super-resolution device, and the multispectral detection component are arranged on the adjustable optical bracket for azimuth adjustment.

[0012] Preferably, it further includes an analysis and control module, which is respectively connected to the hyperspectral imaging component, the piezoelectric ceramic micro-scanning super-resolution device, and the multispectral detection component; the working mode of the analysis and control module is as follows: S1. Execute the infrared detection mode, and control the piezoelectric ceramic micro-scanning super-resolution device to be in a static state; for the same target scene, image acquisition is performed through the mid-infrared beam branch optical path to obtain L frames of multispectral infrared images, where L represents the number of spectral splitting paths of the multi-channel spectral splitting device; S2. Obtain the L frames of multispectral infrared images in the infrared detection mode and perform target detection; S3. If the confidence level of the target detection result is lower than the first confidence level threshold, start the hyperspectral and super-resolution infrared collaborative detection mode, and re-perform image acquisition and detection on the current target scene; if the confidence level of the target detection result is between the first confidence level threshold and the second confidence level threshold, start the infrared super-resolution mode, and re-perform image acquisition and detection on the current target scene; if the confidence level of the target detection result is higher than the second confidence level threshold, output the target detection result; wherein, the first confidence level threshold is less than the second confidence level threshold.

[0013] Preferably, start the hyperspectral and super-resolution infrared collaborative detection mode, and re-perform image acquisition and detection on the current target scene; specifically as follows: Starting the hyperspectral and super-resolution infrared collaborative detection mode includes controlling the piezoelectric ceramic micro-scanning super-resolution device to be in a working state; in the working state, the vibrating component of the piezoelectric ceramic micro-scanning super-resolution device works and causes the output optical path to shift, so that the pixel points on the multi-channel detector perform sub-pixel level shifting according to a preset trajectory during each frame interval; For the current target scene: Continuously collect L×N frames of multispectral infrared images with displacement information through the mid-infrared beam branch optical path, where N represents the number of frames of multispectral infrared images continuously collected for each channel; and synchronously collect M frames of encoded hyperspectral images through the visible / near-infrared light branch optical path, where M represents the number of frames of encoded hyperspectral images continuously collected; Obtain the L×N frames of multispectral infrared images with displacement information, and fuse the multispectral infrared images of the same channel to obtain a fused multispectral infrared image; obtain the M frames of encoded hyperspectral images and perform decoding to obtain a super-resolution spectral image; Perform target detection based on the fused multispectral infrared image and the super-resolution spectral image, obtain the target detection result and output it.

[0014] Preferably, start the infrared super-resolution mode, and re-perform image acquisition and detection on the current target scene; specifically as follows: Start the infrared super-resolution mode, including controlling the piezoelectric ceramic micro-scanning super-resolution device to be in the working state; in the working state, the vibrating component of the piezoelectric ceramic micro-scanning super-resolution device works and causes the output optical path to be displaced, so that the pixel points on the multi-channel detector perform sub-pixel-level displacement according to a preset trajectory during each frame interval; For the current target scene, continuously collect L×N frames of multi-spectral infrared images with displacement information through the mid-infrared beam branching optical path, where N represents the number of frames of multi-spectral infrared images continuously collected in each channel; Obtain L×N frames of multi-spectral infrared images with displacement information, and fuse the multi-spectral infrared images of the same channel to obtain a fused multi-spectral infrared image; Perform target detection based on the fused multi-spectral infrared image, obtain the target detection result and output it.

[0015] Preferably, the first confidence threshold is set to 30%; the second confidence threshold is set to 80%.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention realizes collaborative optimization in various links such as acquisition, encoding, reconstruction, feedback regulation, and data fusion: (1) High imaging accuracy and rich detail restoration: Utilize the high-speed micro-scanning motion component to achieve sub-pixel-level multi-frame acquisition, and then through the multi-frame fusion reconstruction technology, greatly break through the resolution limit of traditional infrared detectors, generate a super-resolution infrared image with richer details, and maintain the authenticity of physical super-resolution; (2) Fine spectral information acquisition: Modulate the incident light using an encoding template, and complete the effective encoding of hyperspectral data through a micromirror array, effectively suppressing the noise introduced during the encoding process, and improving the spectral signal-to-noise ratio and resolution after reconstruction; (3) Real-time adaptive regulation: Monitor the image reconstruction effect in real time and dynamically adjust the micro-scanning parameters, so that the system can maintain the best working state under different light fields and detection environments, and meet the dynamic and variable application requirements; (4) Compact, lightweight and low power consumption: Utilize the wide-spectrum common-aperture optical scheme to share the same acquisition channel and reduce the number of optical components, which not only simplifies the system structure, but also reduces the overall volume and power consumption, and is very suitable for being carried on unmanned aerial vehicles, satellites and other portable platforms. (5) Comprehensive multi-spectral data processing ability: The system integrates data in two major bands of infrared and visible / near-infrared, and through backend data fusion and multi-mode feature extraction, provides sufficient data support for target detection, classification and accurate identification. Description of the Drawings

[0017] Figure 1 It is the system structure diagram of the present invention.

[0018] In the figure: 1 - Coaxial common - aperture objective lens; 2 - Collimator; 3 - Dichroic beam - splitter element; 4 - Coding template; 5 - Hyperspectral imaging component; 6 - Piezo - electric ceramic micro - scanning super - resolution device; 7 - Multispectral detection component. Detailed implementation mode

[0019] The following combines with the attached Figure 1 , and specifically describes the technical solution of the present invention.

[0020] The present invention provides a spectral imaging system based on adaptive super - resolution, including a coaxial common - aperture objective lens 1, a collimator 2, a dichroic beam - splitter element 3, a coding template 4, a hyperspectral imaging component 5, a piezo - electric ceramic micro - scanning super - resolution device 6, and a multispectral detection component 7; The optical axes of the coaxial common - aperture objective lens 1 and the collimator 2 coincide; the coaxial common - aperture objective lens 1 converges the target light to the collimator 2, and the collimator 2 reshapes it into a parallel light beam; the dichroic beam - splitter element 3 divides the parallel light beam into a visible / near - infrared light beam and a mid - infrared light beam according to a preset splitting ratio, forming a visible / near - infrared light branch optical path and a mid - infrared light beam branch optical path; The coding template 4 and the hyperspectral imaging component 5 are sequentially arranged on the visible / near - infrared light branch optical path along the propagation direction of the visible / near - infrared light beam; the coding template 4 modulates the visible / near - infrared light beam at the pixel level through a micromirror array to improve the spectral signal - to - noise ratio and resolution; the hyperspectral imaging component 5 includes a dispersive element and a detector. After the coded visible / near - infrared light beam is separated into spectra of each band by the dispersive element, a coded hyperspectral image is generated on the detector; The piezo - electric ceramic micro - scanning super - resolution device 6 and the multispectral detection component 7 are sequentially arranged on the mid - infrared light beam branch optical path along the propagation direction of the mid - infrared light beam; the multispectral detection component 7 includes a multi - channel beam - splitting device and a multi - channel detector. After the multi - channel beam - splitting device divides the beam bandwidth passing through the piezo - electric ceramic micro - scanning super - resolution device 6 into multiple narrow - band channels, it enters the multi - channel detector to complete the synchronous acquisition of the multi - spectral infrared image.

[0021] In this embodiment, the preset splitting ratio is 50:50.

[0022] In this embodiment, the dispersive element includes a prism or a grating.

[0023] In this embodiment, the multi - channel beam - splitting device includes a multi - channel band - pass filter or a multi - prism beam - splitter.

[0024] In this embodiment, the multi - channel beam - splitting device and the multi - channel detector respectively adopt a four - channel beam - splitting device and a four - channel detector.

[0025] In this embodiment, an adjustable optical bracket is further included. The coaxial common-aperture objective lens 1, the collimator 2, the dichroic beam splitter 3, the coding template 4, the hyperspectral imaging component 5, the piezoelectric ceramic micro-scanning super-resolution device 6, and the multispectral detection component 7 are arranged on the adjustable optical bracket for azimuth adjustment.

[0026] In this embodiment, an analysis and control module is further included. The analysis and control module is respectively connected to the hyperspectral imaging component 5, the piezoelectric ceramic micro-scanning super-resolution device 6, and the multispectral detection component 7. The working mode of the analysis and control module is as follows: S1. Execute the infrared detection mode (coarse detection mode), and control the piezoelectric ceramic micro-scanning super-resolution device 6 to be in a static state. For the same target scene, image acquisition is performed through the mid-infrared beam branch optical path to obtain L frames of multispectral infrared images, where L represents the number of spectral splitting paths of the multi-channel spectral splitting device. S2. Obtain the L frames of multispectral infrared images in the infrared detection mode and perform target detection. S3. If the confidence level of the target detection result is lower than the first confidence level threshold, start the hyperspectral and super-resolution infrared collaborative detection mode, and re-perform image acquisition and detection on the current target scene. If the confidence level of the target detection result is between the first confidence level threshold and the second confidence level threshold, start the infrared super-resolution mode, and re-perform image acquisition and detection on the current target scene. If the confidence level of the target detection result is higher than the second confidence level threshold, output the target detection result. Among them, the first confidence level threshold is less than the second confidence level threshold.

[0027] In addition, the analysis and control module can also be connected to the coding template 4 to dynamically adjust the coding strategy according to the image reconstruction effect (image detection result). Among them, the coding template 4 is a prior art, and for detailed content, refer to the patent application CN107687896A.

[0028] In this embodiment, start the hyperspectral and super-resolution infrared collaborative detection mode, and re-perform image acquisition and detection on the current target scene. Specifically as follows: Starting the hyperspectral and super-resolution infrared collaborative detection mode includes controlling the piezoelectric ceramic micro-scanning super-resolution device 6 to be in a working state. In the working state, the vibrating component of the piezoelectric ceramic micro-scanning super-resolution device 6 works and causes the output optical path to be displaced, so that the pixel points on the multi-channel detector perform sub-pixel level displacement according to a preset trajectory during each frame interval. For the current target scene: Continuously acquire L×N frames of multispectral infrared images with displacement information through the mid-infrared beam branch optical path, where N represents the number of frames of multispectral infrared images continuously acquired for each channel. And synchronously continuously acquire M frames of coded hyperspectral images through the visible light / near-infrared light branch optical path, where M represents the number of frames of coded hyperspectral images continuously acquired. Obtain L×N frames of multi-spectral infrared images with displacement information, and fuse the multi-spectral infrared images of the same channel to obtain a fused multi-spectral infrared image; obtain M frames of encoded hyperspectral images and decode them to obtain a super-resolution spectral image; Perform target detection based on the fused multi-spectral infrared image and the super-resolution spectral image, and obtain and output the target detection result.

[0029] In this embodiment, start the infrared super-resolution mode and re-collect and detect images of the current target scene; specifically as follows: Starting the infrared super-resolution mode includes controlling the piezoelectric ceramic micro-scanning super-resolution device 6 to be in the working state; in the working state, the vibrating component of the piezoelectric ceramic micro-scanning super-resolution device 6 works and causes the output optical path to be displaced, so that the pixel points on the multi-channel detector perform sub-pixel level displacement according to a preset trajectory during each frame interval; For the current target scene, continuously collect L×N frames of multi-spectral infrared images with displacement information through the mid-infrared beam branch optical path, where N represents the number of frames of multi-spectral infrared images continuously collected in each channel; Obtain L×N frames of multi-spectral infrared images with displacement information, and fuse the multi-spectral infrared images of the same channel to obtain a fused multi-spectral infrared image; Perform target detection based on the fused multi-spectral infrared image, and obtain and output the target detection result.

[0030] In this embodiment, the first confidence threshold is set to 30%; the second confidence threshold is set to 80%.

[0031] In summary, through intelligent automatic feedback regulation, this system realizes the self-feedback acquisition of high-spatial-resolution and high-spectral-dimension data of the target of interest. And this system can be widely applied to various scenarios such as military reconnaissance and security, remote sensing and environmental monitoring, airborne platforms and unmanned systems, and industrial inspection and safety monitoring. Through the "coarse first and then fine" adaptive feedback working mode, after quickly scanning and positioning the target area over a large range, it automatically switches to infrared spatial super-resolution and visible spectral band super-resolution imaging to realize the real-time acquisition of high-spatial-resolution and high-spectral-dimension data, which not only greatly reduces the initial data volume but also ensures the fine detection and analysis of key targets.

[0032] The above are the preferred embodiments of the present invention. All changes made according to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention in terms of the functions and effects produced belong to the protection scope of the present invention.

Claims

1. A spectral imaging system based on adaptive super-resolution, characterized in that It includes a coaxial common-aperture objective lens (1), a collimator (2), a dichroic beam splitter (3), a coding template (4), a hyperspectral imaging component (5), a piezoelectric ceramic micro-scanning super-resolution device (6), and a multispectral detection component (7); The optical axes of the coaxial common-aperture objective lens (1) and the collimator (2) coincide; the coaxial common-aperture objective lens (1) converges the target light to the collimator (2), and the collimator (2) reshapes it into a parallel light beam; the dichroic beam splitter (3) splits the parallel light beam into a visible / near-infrared light beam and a mid-infrared light beam according to a preset splitting ratio, forming a visible / near-infrared light branch optical path and a mid-infrared light beam branch optical path; The coding template (4) and the hyperspectral imaging component (5) are sequentially arranged on the visible / near-infrared light branch optical path along the propagation direction of the visible / near-infrared light beam; the coding template (4) performs pixel-level modulation on the visible / near-infrared light beam through a micromirror array to improve the spectral signal-to-noise ratio and resolution; the hyperspectral imaging component (5) includes a dispersive element and a detector. After the coded visible / near-infrared light beam is separated into spectra of each band by the dispersive element, a coded hyperspectral image is generated on the detector; The piezoelectric ceramic micro-scanning super-resolution device (6) and the multispectral detection component (7) are sequentially arranged on the mid-infrared light beam branch optical path along the propagation direction of the mid-infrared light beam; the multispectral detection component (7) includes a multi-channel beam splitter and a multi-channel detector. After the multi-channel beam splitter splits the beam bandwidth passing through the piezoelectric ceramic micro-scanning super-resolution device (6) into multiple narrow-band channels, it enters the multi-channel detector to complete the synchronous acquisition of the multi-spectral infrared image.

2. The spectral imaging system based on adaptive super-resolution according to claim 1, wherein The preset splitting ratio is 50:

50.

3. The spectral imaging system based on adaptive super-resolution according to claim 1, wherein The dispersive element includes a prism or a grating.

4. A spectral imaging system based on adaptive super-resolution according to claim 1, characterized in that, The multi-channel beam splitter includes a multi-channel band-pass filter or a multi-prism beam splitter.

5. The spectral imaging system based on adaptive super-resolution according to claim 1, characterized in that The multi-channel beam splitter and the multi-channel detector respectively adopt a four-channel beam splitter and a four-channel detector.

6. The spectral imaging system based on adaptive super-resolution according to claim 1, wherein It further includes an adjustable optical bracket, and the coaxial common-aperture objective lens (1), the collimator (2), the dichroic beam splitter (3), the coding template (4), the hyperspectral imaging component (5), the piezoelectric ceramic micro-scanning super-resolution device (6), and the multispectral detection component (7) are arranged on the adjustable optical bracket for azimuth adjustment.

7. The spectral imaging system based on adaptive super-resolution according to claim 1, characterized in that It further includes an analysis and control module, and the analysis and control module is respectively connected to the hyperspectral imaging component (5), the piezoelectric ceramic micro-scanning super-resolution device (6), and the multispectral detection component (7); the working mode of the analysis and control module is as follows: S1. Execute the infrared detection mode, and control the piezoelectric ceramic micro-scanning super-resolution device (6) to be in a static state; for the same target scene, image acquisition is performed through the mid-infrared light beam branch optical path to obtain L frames of multi-spectral infrared images, where L represents the number of splitting optical paths of the multi-channel beam splitter; S2. Obtain the L frames of multi-spectral infrared images in the infrared detection mode and perform target detection; S3. If the confidence level of the target detection result is lower than the first confidence level threshold, then start the hyperspectral and super-resolution infrared collaborative detection mode, and re-perform image acquisition and detection on the current target scene; If the confidence level of the target detection result is between the first confidence level threshold and the second confidence level threshold, the infrared super-resolution mode is activated, and the image acquisition and detection of the current target scene are performed again; If the confidence level of the target detection result is higher than the second confidence level threshold, the target detection result is output; where the first confidence level threshold is less than the second confidence level threshold.

8. The spectral imaging system based on adaptive super-resolution according to claim 7, wherein, The hyperspectral and super-resolution infrared collaborative detection mode is activated, and the image acquisition and detection of the current target scene are performed again; specifically as follows: Activating the hyperspectral and super-resolution infrared collaborative detection mode includes controlling the piezoelectric ceramic micro-scanning super-resolution device (6) to be in the working state; in the working state, the vibrating component of the piezoelectric ceramic micro-scanning super-resolution device (6) works and causes the output optical path to be displaced, so that the pixel points on the multi-channel detector perform sub-pixel level displacement according to a preset trajectory during each frame interval; For the current target scene: Continuously collect L×N frames of multi-spectral infrared images with displacement information through the mid-infrared beam branch optical path, where N represents the number of frames of multi-spectral infrared images continuously collected in each channel; and synchronously collect M frames of encoded hyperspectral images through the visible / near-infrared light branch optical path, where M represents the number of frames of encoded hyperspectral images continuously collected; Obtain L×N frames of multi-spectral infrared images with displacement information, and fuse the multi-spectral infrared images of the same channel to obtain a fused multi-spectral infrared image; obtain M frames of encoded hyperspectral images and decode them to obtain a super-resolution spectral image; Perform target detection based on the fused multi-spectral infrared image and the super-resolution spectral image, obtain the target detection result and output it.

9. A spectral imaging system based on adaptive super-resolution according to claim 7, characterized in that, The infrared super-resolution mode is activated, and the image acquisition and detection of the current target scene are performed again; specifically as follows: Activating the infrared super-resolution mode includes controlling the piezoelectric ceramic micro-scanning super-resolution device (6) to be in the working state; in the working state, the vibrating component of the piezoelectric ceramic micro-scanning super-resolution device (6) works and causes the output optical path to be displaced, so that the pixel points on the multi-channel detector perform sub-pixel level displacement according to a preset trajectory during each frame interval; For the current target scene, continuously collect L×N frames of multi-spectral infrared images with displacement information through the mid-infrared beam branch optical path, where N represents the number of frames of multi-spectral infrared images continuously collected in each channel; Obtain L×N frames of multi-spectral infrared images with displacement information, and fuse the multi-spectral infrared images of the same channel to obtain a fused multi-spectral infrared image; Perform target detection based on the fused multi-spectral infrared image, obtain the target detection result and output it.

10. A spectral imaging system based on adaptive super-resolution according to claim 7, characterized in that, The first confidence level threshold is set to 30%; the second confidence level threshold is set to 80%.

Images