Snapshot multispectral imaging device with high dynamic range

Abstract

The invention discloses a high-dynamic-range snapshot multispectral imaging device, which is characterized in that optical signals in a field of view are received through a front optical lens, are converged and then are divided into two paths with different energies through a light splitting device, and then the two paths of optical signals are corrected and converged through two correction mirrors with aperture-adjustable diaphragms, so that the high-dynamic-range snapshot multispectral imaging device is formed. And the two snapshot multispectral detectors at the rear end respectively receive images. And a controller is used for controlling parameters of the two diaphragms and adjusting the light-passing energy ratio of the two paths of light rays. Meanwhile, a controller is used for controlling exposure time and gain parameters of the two snapshot multispectral detectors, and synchronous imaging of the two snapshot multispectral detectors is guaranteed. And the data processor synthesizes the two paths of multispectral images acquired by the two snapshot multispectral detectors under the conditions of different light energies and different imaging parameters into a high-dynamic-range multispectral image and outputs the high-dynamic-range multispectral image.

Description

Technical Field

[0001] This invention relates to a high dynamic range snapshot multispectral imaging device, belonging to the field of multispectral imaging technology. Background Technology

[0002] The integrated image and spectrum characteristic of multispectral imaging technology makes it a widely applicable multidimensional information acquisition technique. Multispectral imaging offers various spectral dispersion methods suitable for different application scenarios. In applications involving changing target scenes, imaging devices need to acquire complete multispectral data simultaneously within a single exposure cycle. Snapshot multispectral imaging, employing a pixel-level filter detector (image sensor) array, integrates a pixel-level filter array onto the detector surface, treating adjacent N×N pixel filter arrays as a spectral unit. Each filter has a different center wavelength, allowing for the simultaneous acquisition of image data from multiple spectral channels. This multispectral imaging method sacrifices spatial resolution to acquire spectral data with a certain level of spatial and spectral resolution, making it particularly suitable for dynamic scene imaging needs and facilitating the miniaturization of multispectral imaging devices.

[0003] Because the light energy received in different spectral bands within a target scene varies significantly, snapshot multispectral imaging using pixel-level filter arrays requires consistent exposure parameters across all channels. This limits the detector's dynamic range, leading to overexposure or underexposure in certain spectral bands. In applications where the spatial morphology, photometric properties, and spectral characteristics of a target change rapidly, multispectral imaging devices must be able to quickly and synchronously acquire multispectral images while maintaining a high dynamic range and obtaining complete photometric variation information for each spectral band.

[0004] Currently, some detectors can acquire high dynamic range (HDR) images in a single exposure using high-low gain synthesis. However, the dynamic range is limited by the detector, making it difficult to cover all spectral channels when there are significant differences between spectral bands. By incorporating a beam splitter in the imaging optical path to divide the optical signal into two paths of different energies, each using a separate detector and controlled with different imaging parameters for synchronous imaging, the two images can be synthesized to generate a HDR image. Alternatively, dividing the optical signal into two parts with different proportions, then receiving and imaging them with detectors, and finally combining the images, can achieve an even greater dynamic range than the original detector's dynamic range.

[0005] For snapshot multispectral imaging, the optical signal can be split into two signals of different energies, which are then received and imaged by two snapshot multispectral detectors with different imaging parameters. The two images are then combined to generate a high dynamic range snapshot multispectral image. This imaging scheme can obtain multispectral images with a larger dynamic range, while snapshot imaging ensures the synchronicity of spectral information acquisition, and can be used to acquire information on rapidly changing target spatial morphology, photometric properties, and spectral characteristics. Summary of the Invention

[0006] The technical problem to be solved by this invention is that, in snapshot multispectral imaging, the dynamic range of a single detector is difficult to meet the brightness range of each spectral band due to the different light intake of multiple spectral channels. This invention proposes a high dynamic range snapshot multispectral imaging device by using dual-channel independent snapshot multispectral detectors to simultaneously acquire images with different imaging parameters after the light is split at different ratios at the rear of the lens, and then synthesizing the images with different imaging parameters of the dual channels to achieve high dynamic range imaging.

[0007] The technical solution adopted in this invention is: a high dynamic range snapshot multispectral imaging device, comprising a front optical lens, a beam splitter, a first correction mirror, a second correction mirror, a first detector, a second detector, a controller, and a data processor;

[0008] The front optical lens receives light signals within the field of view, which are then converged and split into two paths of different energy by a beam splitter. The two light signals of different energy are converged by the first and second correction mirrors at the rear end and received by the first and second detectors, respectively. The controller controls the parameter adjustment of the first and second detectors and controls the first and second detectors to synchronize exposure and imaging. After receiving the light signals, the first and second detectors convert them into multispectral image data and send them to the data processor. The data processor combines the multispectral image data output by the first and second detectors into a new multispectral image and outputs it.

[0009] Furthermore, the beam splitter allows the reflected light and transmitted light to have different proportions, splitting the light output from the front optical lens into two paths of high energy and low energy.

[0010] Furthermore, the first and second correction mirrors integrate an aperture diaphragm adjustment module for adjusting the amount of light transmitted. The aperture diaphragm adjustment module is controlled by a controller to adjust the aperture parameters. After the two beams output by the beam splitter are adjusted by the aperture diaphragm adjustment modules of the first and second correction mirrors to adjust different proportions of light transmitted, the energy ratio of the high-energy and low-energy beams increases.

[0011] Furthermore, the first and second detectors employ CMOS image sensors and integrate pixel-level filter arrays on the detector surface to simultaneously obtain spectral data in the visible and near-infrared bands of multiple channels.

[0012] Furthermore, the pixel-level filter array adopts 3×3, 4×4 or 5×5 pixel filter array units.

[0013] Furthermore, the first and second detectors adopt an externally triggered exposure mode, with the controller controlling the exposure synchronization and gain parameters; the controller sends a trigger signal to ensure that the first and second detectors have the same imaging frame rate, independent exposure time, and consistent exposure center time.

[0014] Furthermore, the data processor receives the raw image data output by the first detector and the second detector, and according to the pixel correspondence between the first detector and the second detector, uses the conversion coefficients of the gray values ​​of the two channels under different combinations of aperture parameters, gain parameters, and exposure time parameters to merge the images of the low-energy channel and the high-energy channel of each pixel unit under different imaging parameters and convert them into a new image.

[0015] Furthermore, the data processor identifies pixel units in the high-energy channel image whose grayscale values ​​exceed a threshold, generates new grayscale values ​​from the corresponding pixel units in the low-energy channel image according to a conversion coefficient, and replaces the grayscale values ​​of the pixel units whose grayscale values ​​exceed the threshold with the new grayscale values ​​to synthesize a new image.

[0016] Furthermore, the data processor identifies pixel units in the low-energy channel image whose grayscale value is lower than a threshold, generates a new grayscale value from the grayscale value of the corresponding pixel unit in the high-energy channel image according to a conversion coefficient, and replaces the grayscale value of the pixel unit whose grayscale value is lower than the threshold with the new grayscale value to synthesize a new image.

[0017] Furthermore, the high dynamic range snapshot multispectral imaging device operates in the visible and near-infrared spectral band of 0.4 micrometers to 1.0 micrometers.

[0018] The advantages of this invention compared to the prior art are:

[0019] This invention proposes a high dynamic range snapshot multispectral imaging device. After the light is split at different ratios at the rear of the lens, dual-channel independent snapshot multispectral detectors are used to simultaneously acquire images and adjust different imaging parameters. The dual-channel images with different imaging parameters are obtained and high dynamic range imaging is achieved through image synthesis. It takes into account both high dynamic range and simultaneous acquisition of multispectral images, can adapt to a wider brightness range in each spectral band, and is more suitable for applications where the spatial morphology, photometric characteristics, and spectral features of the target are dynamically changing. Attached Figure Description

[0020] Figure 1 A diagram of a high dynamic range snapshot multispectral imaging system;

[0021] Figure 2 This is a schematic diagram of dual-channel exposure triggering for a high dynamic range snapshot multispectral imaging system. Detailed Implementation

[0022] The present invention will be described in conjunction with the accompanying drawings.

[0023] like Figure 1 As shown, a high dynamic range snapshot multispectral imaging device includes a front optical lens, a beam splitter, a first correction mirror, a second correction mirror, a first detector, a second detector, a controller, and a data processor.

[0024] The front optical lens receives light signals within the field of view, which are then converged and split into two paths of different energy by the beam splitter. These paths are then converged again by the first and second correction mirrors at the rear end and received by the first and second detectors. The controller adjusts the parameters of the first and second detectors and synchronizes their exposure and imaging. After receiving the light signals, the first and second detectors convert them into multispectral image data and send it to the data processor. The data processor then combines the multispectral image data output from the two detectors into a high dynamic range multispectral image for output.

[0025] The high dynamic range snapshot multispectral imaging device operates in the visible and near-infrared spectral band from 0.4 micrometers to 1.0 micrometers.

[0026] In this embodiment of the invention, the front optical lens, beam splitter, first correction mirror, second correction mirror, first detector, and second detector include matching component mechanical structures and provide mounting positions for the controller and data processor.

[0027] The aforementioned beam splitter separates a light signal containing visible and near-infrared spectral bands into two paths with different energies. The first and second correction mirrors correct and converge the light output from the beam splitter, which is then received and imaged by the first and second detectors. The first and second correction mirror assemblies integrate an aperture diaphragm adjustment module for adjusting the light transmission. The controller controls the aperture adjustment of the first and second correction mirror assemblies.

[0028] In this embodiment of the invention, the beam splitter uses a beam-splitting prism, splitting the optical signal into two paths at a 9:1 ratio. The optical system design ensures that the aperture stops for the two paths are positioned near the first and second corrective mirrors, respectively. The first and second corrective mirror assemblies integrate an aperture stop adjustment module to adjust the light transmission. By adjusting the aperture stop parameters, the ratio of the light transmission of the two stops is made to be 1 to 8 times, thereby increasing the energy ratio of the high-energy and low-energy light paths.

[0029] The first and second detectors receive two optical signals with different energies, respectively. The first and second detectors integrate pixel-level filter arrays on their surfaces, simultaneously acquiring image data from multiple spectral channels. The first and second detectors employ CMOS image sensors, operating in the visible and near-infrared spectral bands. The pixel-level filter arrays of the first and second detectors utilize 3×3, 4×4, or 5×5 pixel filter array units. The first and second detector assemblies include driving circuitry and signal processing circuitry for driving and controlling the imaging and data output of the CMOS image sensor.

[0030] In this embodiment of the invention, the first and second detectors are snapshot multispectral detectors that integrate a 4×4 pixel-level filter array onto the surface of a 2048×1088 pixel CMOS image sensor, simultaneously acquiring spectral data from 16 channels in a single imaging operation. Both the first and second detectors are global electronic shutter detectors.

[0031] The first and second detectors employ an externally triggered exposure mode, with the controller synchronizing their exposure. The controller sends trigger signals to ensure that the frame rate and exposure center time of the first and second detectors are consistent. The controller can control the first and second detectors to have different exposure times. The controller also sets the gain parameters for the first and second detectors.

[0032] In this embodiment of the invention, the controller controls the frame rate and exposure time ratio of the two detectors according to imaging requirements. The controller continuously controls the two detectors according to a fixed trigger signal to ensure that the image data acquisition time of the two detectors remains synchronized. The controller synchronously outputs exposure trigger pulse signals to the first and second detectors according to the set frame rate and exposure time. The frame rates of the first and second detectors are consistent, and the exposure center times of the first and second detectors are kept consistent during each frame image acquisition. The exposure times of the first and second detectors can be set to different times. The exposure trigger timing is as follows: Figure 2 As shown, the exposure pulse trigger uses a level triggering method. During the high-level period, the detector exposes and forms an image. The controller sets the center time of the high-level pulses of the two channels to be consistent. The controller controls the exposure time ratio of the first and second detectors to be 1 to 8 times. The controller sets the gain parameters of the first and second detectors. Simultaneously, the controller is responsible for external communication control and power conversion, ensuring the power supply and operating parameter control of the first and second detectors.

[0033] In this embodiment of the invention, the beam splitting ratio of the two channels of the beam splitter and the light transmission adjustment ratio of the apertures of the two correction mirror assemblies determine the intensity of the light signal entering the two detectors. Due to the limited beam splitting ratio of the beam splitter prism and the aperture adjustment ratio, the acquisition of the light signal is further controlled by adjusting the exposure time and gain to adapt to a wider range of brightness. In the two channels, the high-energy channel uses a high beam splitting ratio and a high light transmission aperture adjustment setting, combined with a long exposure time and high gain parameter imaging data acquisition, which can adapt to low-brightness regions and spectral bands in the scene target. The low-energy channel uses a low beam splitting ratio channel and a low light transmission aperture adjustment setting, combined with a short exposure time and low gain parameter imaging data acquisition, which can adapt to high-brightness regions and spectral bands in the scene target. Depending on the application scenario, some parameters such as the aperture, exposure time, and gain of the two channels can be set to be consistent during operation.

[0034] The data processor receives raw image data output from the first and second detectors. Based on the pixel correspondence between the two detectors, it merges the grayscale values ​​of the low-energy and high-energy channels of each pixel unit under different imaging parameters, converting them into new image grayscale values. The data processor performs conversion and synthesis based on the conversion coefficients of the grayscale values ​​of the two channels under different calibrated combinations of aperture adjustment parameters, gain parameters, and exposure time parameters, outputting a high dynamic range snapshot multispectral image. The data processor synthesizes the image by replacing pixel units in the high-energy channel image that exceed a grayscale threshold according to the conversion coefficients: it identifies pixel units in the high-energy channel image whose grayscale values ​​exceed the threshold, replaces them with the grayscale values ​​of the corresponding pixel units in the low-energy channel image generated according to the conversion coefficients, and synthesizes a new image; or it synthesizes the image by replacing pixel units in the low-energy channel image that are below the grayscale threshold according to the conversion coefficients: it identifies pixel units in the low-energy channel image whose grayscale values ​​are below the threshold, replaces them with the grayscale values ​​of the corresponding pixel units in the high-energy channel image generated according to the conversion coefficients, and synthesizes a new image.

[0035] In this embodiment of the invention, the data processor receives image data output from two snapshot multispectral detectors and synthesizes the grayscale values ​​of the low-energy and high-energy channels of each pixel unit under different imaging parameters before outputting the result. First, the conversion coefficients of the grayscale values ​​of the high-energy and low-energy channels under different combinations of aperture adjustment parameters, gain parameters, and exposure time parameters are calibrated. Then, the two images are synthesized based on these conversion coefficients. When synthesizing by replacing pixel units exceeding the grayscale threshold in the high-energy channel image according to the conversion coefficients, a threshold is set for the high-energy channel image data. Pixel image data below the grayscale threshold is retained, while pixel image data above the threshold is obtained by calculating the grayscale value of the corresponding pixel image in the low-energy channel. Assume the grayscale value of the low-energy channel image is D. L The grayscale value of the high-energy channel image is D. H The grayscale values ​​of the synthesized image are:

[0036]

[0037] Where: a is the conversion ratio coefficient, determined by the beam splitting ratio, the light intake ratio of the aperture adjustment, the gain coefficient ratio, and the exposure time ratio; b is the conversion offset coefficient, determined by the output values ​​of the two channel detectors when there is no light and the conversion ratio coefficient a.

[0038] When compositing by replacing pixels below a grayscale threshold in the low-energy channel image with conversion coefficients, a threshold is set for the low-energy channel image data. Pixels above the grayscale threshold are retained, while pixels below the threshold are calculated using the corresponding grayscale values ​​from the high-energy channel image to obtain the new image's grayscale value. Assume the grayscale value of the low-energy channel image is D. L The grayscale value of the high-energy channel image is D. H The grayscale values ​​of the synthesized image are:

[0039]

[0040] Where: c is the conversion ratio coefficient, determined by the beam splitting ratio, the aperture adjustment light intake ratio, the gain coefficient ratio, and the exposure time ratio. d is the conversion offset coefficient, determined by the output values ​​of the two channel detectors when there is no light and the conversion ratio coefficient c.

[0041] In this embodiment of the invention, the data processing outputs a synthesized high dynamic range snapshot multispectral image. The data processing can also simultaneously output the original snapshot multispectral images of the two detectors for subsequent high dynamic range image synthesis processing, depending on the needs of use.

[0042] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

[0043] The parts of this invention not described in detail are well-known to those skilled in the art.

Claims

1. A high dynamic range snapshot multispectral imaging device, characterized in that: It includes a front-facing optical lens, a beam splitter, a first correction mirror, a second correction mirror, a first detector, a second detector, a controller, and a data processor; The front optical lens receives light signals within the field of view, which are then converged and split into two paths of different energy by a beam splitter. The two light signals of different energy are converged by the first and second correction mirrors at the rear end and received by the first and second detectors, respectively. The controller controls the parameter adjustment of the first and second detectors and controls the first and second detectors to synchronize exposure and imaging. After receiving the light signals, the first and second detectors convert them into multispectral image data and send them to the data processor. The data processor combines the multispectral image data output by the first and second detectors into a new multispectral image and outputs it.

2. The high dynamic range snapshot multispectral imaging device according to claim 1, characterized in that: The beam splitter divides the reflected and transmitted light into two beams of different proportions, separating the light output from the front optical lens into high-energy and low-energy beams.

3. The high dynamic range snapshot multispectral imaging device according to claim 1, characterized in that: The first and second correction mirrors integrate an aperture diaphragm adjustment module for adjusting the amount of light transmitted. The aperture diaphragm adjustment module is controlled by a controller to adjust the aperture parameters. After the two beams output by the beam splitter are adjusted by the aperture diaphragm adjustment modules of the first and second correction mirrors to adjust different proportions of light transmitted, the energy ratio of the high-energy and low-energy beams increases.

4. The high dynamic range snapshot multispectral imaging device according to claim 1, characterized in that: The first and second detectors employ CMOS image sensors and integrate pixel-level filter arrays on the detector surface to simultaneously obtain spectral data in the visible and near-infrared bands of multiple channels.

5. The high dynamic range snapshot multispectral imaging device according to claim 4, characterized in that: The pixel-level filter array uses 3×3, 4×4 or 5×5 pixel filter array units.

6. The high dynamic range snapshot multispectral imaging device according to claim 4, characterized in that: The first and second detectors adopt an externally triggered exposure mode, and the controller controls the exposure synchronization and gain parameters. The controller sends a trigger signal to ensure that the first and second detectors have the same imaging frame rate, independent exposure time, and consistent exposure center time.

7. The high dynamic range snapshot multispectral imaging device according to claim 1, characterized in that: The data processor receives the raw image data output by the first and second detectors. Based on the pixel correspondence between the first and second detectors, and using the conversion coefficients of the grayscale values ​​of the two channels under different combinations of aperture parameters, gain parameters, and exposure time parameters, it merges the images of the low-energy channel and high-energy channel of each pixel unit under different imaging parameters and converts them into a new image.

8. The high dynamic range snapshot multispectral imaging device according to claim 7, characterized in that: The data processor identifies pixel units in the high-energy channel image whose grayscale values ​​exceed a threshold. It then generates new grayscale values ​​for the corresponding pixel units in the low-energy channel image using a conversion coefficient. Finally, it replaces the grayscale values ​​of the pixel units whose grayscale values ​​exceed the threshold with the new grayscale values ​​to synthesize a new image.

9. A high dynamic range snapshot multispectral imaging device according to claim 7, characterized in that: The data processor identifies pixel units in the low-energy channel image whose grayscale value is lower than a threshold, generates a new grayscale value from the corresponding pixel unit in the high-energy channel image according to a conversion coefficient, and replaces the grayscale value of the pixel unit whose grayscale value is lower than the threshold with the new grayscale value to synthesize a new image.

10. A high dynamic range snapshot multispectral imaging device according to claim 1, characterized in that: The high dynamic range snapshot multispectral imaging device operates in the visible and near-infrared spectral band from 0.4 micrometers to 1.0 micrometers.

Images