High-dynamic optical spectral imaging system and imaging method suitable for dynamic scene

Abstract

The invention discloses a high-dynamic optical spectral imaging system and imaging method suitable for a dynamic scene, belongs to a photoelectric sensor imaging system, solves the problem of complicated operation of a high-dynamic imaging system, and comprises a front lens module, a variable-ratio splitter module, a variable integral exposure module and an imaging module, wherein the front lens module is used for adjusting an aperture and a focal length, the imaging system generally adopts a rear light splitting single-camera system form, and a main body adopts a set of front imaging optical-mechanical system to adapt to the limitation of a large-aperture camera on weight, cost and structural complexity; the imaging module adopts two paths of sensors to image a motion scene at the same time, so that registration and image artifacts caused by multi-frame imaging are avoided; a flexible acquisition strategy is introduced into the imaging method, and each path of sensor enables the capturable brightness intervals of the sensors to be overlapped through variable integral time exposure, so that the expansion of the dynamic range of the system is completed.

Description

Technical Field

[0001] The present invention belongs to a photoelectric sensor imaging system, and in particular relates to a high-dynamic optical spectroscopic imaging system and an imaging method suitable for dynamic scenes. Background Art

[0002] Under high-contrast lighting conditions, the dynamic range of the scene often does not match the dynamic range that the imaging system can record, which brings difficulties to later image processing and scene recognition. In order to solve this problem, high dynamic imaging technology has been studied for many years to overcome the contradiction between the high dynamic range (HDR) characteristics of real scenes and the low dynamic range (LDR) limitations of cameras. At present, high dynamic optical imaging technology is mainly divided into the following three forms:

[0003] (I) High dynamic imaging based on image sensor hardware mainly focuses on reducing noise, improving quantum efficiency, increasing full well charge and other measures to directly improve the dynamic range performance of camera image sensors. This form can inherit the existing imaging optical and mechanical structure, is directly installed, and is easy to use. However, it is also restricted by factors such as device materials, design level, semiconductor production equipment and process level, as well as high prices.

[0004] (ii) High dynamic imaging based on multiple exposures of a single camera mainly uses a camera to continuously shoot a sequence of images with different exposures, and uses brightness mapping between images to fuse the effective areas of the images with different exposures to obtain high dynamic images. This method does not have high requirements on equipment, but requires the scene to remain stable and be shot in a short time, so the application effect in dynamic scenes is poor.

[0005] (III) High dynamic imaging based on multiple image sensors mainly includes multi-view camera array system, front-end spectroscopic multi-camera system and rear-end spectroscopic single-camera system. The multi-view camera array system uses multiple cameras to image the scene at the same time, and has the disadvantage of parallax artifacts caused by spatial position differences; the spectroscopic device of the front-end spectroscopic multi-camera system is located at the front end of the entire system, and multiple cameras receive different branch light path energies respectively, which has the disadvantages of complex system structure, high cost, and is not suitable for large-aperture imaging systems; the rear-end spectroscopic single-camera system uses a set of front-end optical-mechanical structures, and the spectroscopic device is located at the back end of the system. Multiple image sensors receive different branch light path energies respectively, which has the advantages of relatively simple system structure, low cost, and is suitable for dynamic scene imaging and large-aperture imaging system manufacturing. However, the existing rear-end spectroscopic single-camera systems mostly use a single light path form, which is prone to the problem of incomplete coverage of the scene dynamic range, resulting in scene information loss and inability to restore scene information.

[0006] Through the analysis of the above-mentioned current state of the art, it can be seen that current imaging applications often face the problem of too large a dynamic range of scenes, and it is difficult for imaging devices to present all the information in a scene in one image. Single-camera multiple-exposure imaging sacrifices real-time imaging in exchange for an expansion of the dynamic range, which is prone to artifacts caused by post-registration and is difficult to adapt to the situation of scene target motion and perspective changes. High-dynamic image sensor hardware solutions are limited by the existing research and development level, technical barriers and high prices, and multi-view camera array systems and front-end spectroscopic multi-camera systems face the problems of complex systems, high manufacturing costs and difficulty in adapting to large-aperture imaging systems. Summary of the invention

[0007] In order to solve the problem that the hardware solution of high dynamic image sensor is limited by the existing research and development level and high price, and the multi-view camera array system and the front-end spectroscopic multi-camera system face the problems of complex system, high manufacturing cost and difficulty in adapting to large-aperture imaging system, while the rear-end spectroscopic single-camera system mostly adopts a single optical path, which is prone to the problem of incomplete coverage of the scene dynamic range, resulting in loss of scene information and inability to restore scene information, the present invention provides a high dynamic optical spectroscopic imaging system and imaging method suitable for dynamic scenes.

[0008] The inventive concept of the present invention is to abandon the complex multi-camera system form and adopt a system form in which a rear two-way splitter is integrated in a single-lens camera, thereby avoiding the problems of a significant increase in system structure complexity and excessively high costs, and being suitable for the manufacture of large-aperture imaging systems.

[0009] In order to achieve the above objectives and complete the above inventive concept, the present invention adopts the following technical solutions:

[0010] A high dynamic optical spectroscopic imaging system suitable for dynamic scenes includes a front lens module in the form of a rear spectroscopic single camera system, and is special in that it also includes:

[0011] A variable beam splitter module, an imaging module, and a variable integral exposure module connected to the imaging module; the front lens module, the variable beam splitter module, and the imaging module are arranged in sequence along the optical path;

[0012] The front lens module is used to adjust the aperture and focal length and collect scene light beams for the target area; the main body uses a set of front imaging optical-mechanical systems to adapt to the restrictions on weight, cost and structural complexity of large-aperture cameras;

[0013] The variable beam splitter module includes a beam splitter prism or a neutral density filter, which is used to split the scene light beam according to the light flux of m:1 into a high-brightness light beam and a low-brightness light beam, and change the light beam transmission direction to achieve dual-light path imaging of the same scene and the same viewing angle, meet the needs of dynamic scene imaging, overcome the difficulty of image registration, and obtain two imaging light beams;

[0014] The imaging module includes a high-brightness sensor and a low-brightness sensor. An image sensor with a larger light transmittance is called a high-brightness (HE) sensor. Its imaging result is brighter overall, which can prevent the dark environment in the scene from being affected by noise. An image sensor with a smaller light transmittance is called a low-brightness (LE) sensor. Its imaging result is darker overall, which can suppress overexposure of high-brightness scenes. The high-brightness sensor and the low-brightness sensor are used to receive high-brightness light beams and low-brightness light beams, respectively, and perform exposure imaging to output a multi-exposure image sequence of the target area. By simultaneously imaging with the high-brightness sensor and the low-brightness sensor, registration and image artifacts caused by multi-frame imaging can be avoided.

[0015] The variable integral exposure module includes a data processing controller, which is respectively connected to a high brightness sensor and a low brightness sensor, and is used to control synchronous triggering or asynchronous triggering of the high brightness sensor and the low brightness sensor, and control the exposure time of the high brightness sensor and the low brightness sensor.

[0016] The present invention discloses a high-dynamic optical spectroscopic imaging system suitable for dynamic scenes, which abandons the complex multi-camera system form and selects a system form in which a rear two-way spectroscopic system is integrated in a single-lens camera, thereby avoiding the problems of a significant increase in the complexity of the system structure and excessively high costs, and is suitable for the manufacture of large-aperture imaging systems.

[0017] The value range of m is 1≤m≤20.

[0018] At the same time, the present invention also provides a high dynamic optical spectroscopic imaging method suitable for dynamic scenes, using the above-mentioned high dynamic optical spectroscopic imaging system suitable for dynamic scenes, which is special in that it includes the following steps:

[0019] S1, the radiance range of the measurement scene, denoted as L min ~L max , and collect the scene of the target area through the front lens module to obtain the scene light beam;

[0020] S2, splitting the scene beam into a high-brightness beam and a low-brightness beam with a luminous flux ratio of m:1 through a variable splitter module, and calculating the minimum radiance of the high-brightness sensor and the low-brightness sensor in combination with the radiance range of the scene obtained in S1;

[0021] S3. Determine the exposure time t of the highlight sensor HE and the exposure time t of the low-light sensor LE ;

[0022] S4, calculating the maximum radiance value of the high brightness sensor and the low brightness sensor when the pixels are saturated, and combining the minimum radiance of the high brightness sensor and the low brightness sensor, calculating the radiance range actually received by the high brightness sensor and the low brightness sensor, and the radiance range of the actual scene captured; at the same time, obtaining the virtual imaging optical path according to the exposure time of the high brightness sensor and the low brightness sensor;

[0023] S5, determining whether the radiance range actually received by the high brightness sensor and the low brightness sensor completely covers the radiance range of the scene and whether there is an overlapping area in the actual radiance range captured by the high brightness sensor and the low brightness sensor;

[0024] If both conditions are met, the variable integral exposure module controls the high-brightness sensor and the low-brightness sensor to receive the high-brightness light beam and the low-brightness light beam according to the exposure time, and performs exposure imaging to obtain a multi-exposure image sequence of the target area, thereby completing high-dynamic optical spectroscopic imaging of the scene;

[0025] Otherwise, execute S6;

[0026] S6. Extend the exposure time of the low-brightness sensor, use the virtual imaging optical path to compensate for the radiance range actually received by the low-brightness sensor, calculate the actual scene radiance range captured by the high-brightness sensor and the low-brightness sensor, and determine whether the captured actual scene radiance range can fully cover the scene dynamic range;

[0027] If yes, the variable integral exposure module controls the high brightness sensor and the low brightness sensor to receive the high brightness light beam and the low brightness light beam respectively according to the currently determined exposure time, and performs exposure imaging to obtain a multi-exposure image sequence of the target area, thereby completing high dynamic optical spectroscopic imaging of the scene;

[0028] Otherwise, return to S4, adjust the exposure time of the high-brightness sensor, and update the exposure time of the low-brightness sensor to the extended exposure time of the low-brightness sensor, until a multi-exposure image sequence of the target area is obtained, completing the high-dynamic optical spectroscopic imaging of the scene.

[0029] Furthermore, S2 is specifically:

[0030] The scene beam is divided into a high-brightness beam and a low-brightness beam with a luminous flux ratio of m:1 by the variable splitter module, and the minimum radiance L of the high-brightness sensor is calculated by the following formulas in combination with the radiance range of the scene obtained by S1: HE_min , the minimum radiance L of the low-light sensor LE_min :

[0031]

[0032] Among them, L min Indicates the minimum radiance of the scene.

[0033] Furthermore, the specific process of determining the exposure time of the high brightness sensor and the low brightness sensor in S3 is: taking the longest integration time when the signal-to-noise ratio SNR of the dark part signal of the scene is greater than 1 as the exposure time t of the high brightness sensor HE ; Take the inherent minimum integration time of the low-light sensor as the exposure time t of the low-light sensor LE .

[0034] Furthermore, the maximum radiance value when the pixels of the high-brightness sensor and the low-brightness sensor are saturated is calculated in S4, and combined with the minimum radiance of the high-brightness sensor and the low-brightness sensor, the radiance range of the actual scene captured is calculated as follows:

[0035] Calculate the maximum radiance value L when the highlight sensor pixel is saturated HE_max The formula is:

[0036]

[0037] Among them, DN HE_max is the maximum pixel value of the highlight sensor; S HE_max is the number of electrons excited in the highlight sensor; C is the conversion efficiency; A is the photosensitive area of ​​the pixel of the front lens module; ε is the shielding coefficient; F is the ratio of the equivalent focal length and optical aperture of the combination of the front lens module and the variable diffraction module; h is the Planck constant; c is the speed of light; λ is the median wavelength of the incident light; τ is the transmittance of the combination of the front lens module and the variable diffraction module; η qH is the quantum efficiency of the high brightness sensor, t HE is the exposure time of the highlight sensor;

[0038] The maximum radiance value L when the low-light sensor pixel is saturated LE_max The calculation formula is:

[0039]

[0040] Among them, DN LE_max is the maximum pixel value of the low-light sensor; S LE_max is the number of electrons excited in the low-light sensor; t LE is the exposure time of the low-light sensor, η qL is the quantum efficiency of the low-light sensor;

[0041] Determine the radiance range actually received by the highlight sensor as:

[0042] △L HE_accept =L HE_min ~L HE_max ;

[0043] Divided by the splitting ratio, the actual scene radiance range captured by the highlight sensor is:

[0044]

[0045] Determine the actual radiance range received by the low-light sensor: △L LE_accept =L LE_min ~L LE_max ;

[0046] Divided by the splitting ratio, the actual scene radiance range captured by the low-light sensor is:

[0047]

[0048] Furthermore, the virtual imaging light path in S4 includes a high-brightness virtual light path and a low-brightness virtual light path.

[0049] Furthermore, in S5, the expression for judging whether the radiance range actually received by the high brightness sensor and the low brightness sensor completely covers the radiance range of the scene is:

[0050]

[0051] The expression for judging whether the actual radiance range of the high brightness sensor and the low brightness sensor has an overlapping area is:

[0052]

[0053] Furthermore, the specific process of extending the exposure time of the low-brightness sensor in S6 and using the virtual imaging optical path to compensate for the radiance range actually received by the low-brightness sensor is as follows:

[0054] Take the exposure time of the low-light sensor extended for the i-th time as a represents the ratio of the exposure time of the low-light sensor extended for the i-th time to the exposure time of the low-light sensor before the extension, where a≥1.1;

[0055] The radiance range actually received by the virtual imaging optical path for:

[0056]

[0057] in, represents the minimum radiance of the virtual imaging light path obtained by changing the exposure time for the i-th time, represents the maximum radiance of the virtual imaging light path obtained by changing the exposure time for the i-th time;

[0058] Determine whether the radiance range actually received by the virtual imaging optical path overlaps with the radiance range actually received by the low-brightness sensor. The judgment formula is as follows:

[0059] Is it

[0060] If not There is overlap, and the radiance range actually received by the low-brightness sensor becomes:

[0061]

[0062] If If there is no overlap, return to modify and extend the exposure time of the low-light sensor.

[0063] Furthermore, the exposure time t of the highlight sensor is adjusted in S6. HE Specifically: Reduce the exposure time of the high-brightness sensor exponentially.

[0064] Furthermore, the instrument for measuring the radiance range of the scene in S1 is a radiometer.

[0065] Beneficial effects of the present invention:

[0066] 1. The present invention discloses a high-dynamic optical spectroscopic imaging system suitable for dynamic scenes. It abandons the complex multi-camera system form and selects a system form in which a rear two-way spectroscopic system is integrated in a front lens module, thereby avoiding the problem of a substantial increase in system structure complexity and excessively high costs, and is suitable for the manufacture of large-aperture imaging systems.

[0067] 2. In a high-dynamic optical spectroscopic imaging system suitable for dynamic scenes, a high-brightness sensor and a low-brightness sensor are used to simultaneously image the moving scene, thereby avoiding registration and image artifacts caused by multi-frame imaging;

[0068] 3. In a high-dynamic optical spectroscopic imaging method suitable for dynamic scenes of the present invention, the radiance range of the scene that can be actually obtained by each sensor is controlled by adjusting the spectroscopic ratio, and the expansion of the first-level dynamic range is achieved through the "optical lever" effect;

[0069] 4. In a high dynamic optical spectroscopic imaging method suitable for dynamic scenes, a flexible acquisition strategy is introduced. Each sensor makes the capture brightness interval of each sensor overlap by variable integral time exposure, thus completing the expansion of the secondary dynamic range of the system and preventing the loss of scene information.

[0070] 5. In a high-dynamic optical spectroscopic imaging method suitable for dynamic scenes of the present invention, two sensors can adopt an asynchronous acquisition method to multiply the system frame rate, thereby improving the sampling efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 This is a schematic diagram of the structure of Embodiment 1 of a high dynamic optical spectroscopic imaging system suitable for dynamic scenes according to the present invention;

[0072] Figure 2 This is a schematic diagram of the imaging principle and results of Example 1 of a high dynamic optical spectroscopic imaging system suitable for dynamic scenes of the present invention;

[0073] Figure 3 This is a flow chart of S1 to S5 in Example 1 of a high dynamic optical spectroscopic imaging method suitable for dynamic scenes of the present invention;

[0074] Figure 4 A schematic diagram of adding a virtual imaging optical path to supplement the missing radiance interval in S6 in Example 1 of a high dynamic optical spectroscopic imaging method suitable for dynamic scenes of the present invention;

[0075] Figure 5 This is a schematic diagram of an image frame sequence obtained by a synchronous acquisition working mode in Embodiment 1 of a high dynamic optical spectroscopic imaging method suitable for dynamic scenes of the present invention;

[0076] Figure 6 This is the asynchronous acquisition working mode in Embodiment 2 of a high dynamic optical spectroscopic imaging method suitable for dynamic scenes of the present invention;

[0077] Figure 7 It is a schematic diagram of the radiance area loss caused by adopting the comparative example of the present invention without adjusting the integration time. DETAILED DESCRIPTION

[0078] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] Example 1

[0080] The embodiment of the present invention provides a high dynamic optical spectroscopic imaging system suitable for dynamic scenes, such as Figure 1 As shown, including:

[0081] The front lens module adopts the form of a rear-split single camera system, which is used to adjust the aperture and focal length and collect scene light beams for the target area; the main body uses a front imaging optical-mechanical system to adapt to the limitations of large-aperture cameras on weight, cost and structural complexity.

[0082] The variable beam splitter module includes a beam splitter prism or a neutral density filter, which is used to split the scene light beam according to the m:1 luminous flux (1≤m≤20) into a high-brightness beam and a low-brightness beam, and change the beam transmission direction to achieve dual-light path imaging of the same scene at the same viewing angle, meet the needs of dynamic scene imaging, and overcome the difficulty of image registration.

[0083] The imaging module includes a high-brightness sensor and a low-brightness sensor. The image sensor with a larger light transmittance is called a high-brightness (HE) sensor. Its imaging result is brighter overall, which can prevent the dark environment in the scene from being affected by noise; the image sensor with a smaller light transmittance is called a low-brightness (LE) sensor. Its imaging result is darker overall, which can suppress the overexposure of the high-brightness scene. The high-brightness sensor and the low-brightness sensor are used to receive the high-brightness light beam and the low-brightness light beam respectively, and perform exposure imaging to output a multi-exposure image sequence of the target area. In this embodiment, the imaging module is connected to a memory, and the output multi-exposure image sequence of the target area can be stored to facilitate the synthesis of the multi-exposure image sequence of the target area.

[0084] In this embodiment, the high-brightness sensor and the low-brightness sensor perform imaging simultaneously, which can avoid registration and image artifacts caused by multi-frame imaging.

[0085] The variable integral exposure module includes a data processing controller, which is respectively connected to a high brightness sensor and a low brightness sensor, and is used to control synchronous triggering or asynchronous triggering of the high brightness sensor and the low brightness sensor, and control the exposure time of the high brightness sensor and the low brightness sensor.

[0086] In this embodiment, the imaging system generally adopts the form of a rear-splitting single camera system, and the main body uses a set of front-mounted imaging optical and mechanical systems to adapt to the restrictions of large-aperture cameras on weight, cost and structural complexity. The scene light beam in the target area is input into the variable spectroscope module, and the light flux splitting ratio is m:1 (m>1), and the light beam transmission direction is changed to obtain two imaging light beams. By splitting into two imaging light beams, dual optical paths can be used to image the same scene at the same viewing angle, meeting the needs of dynamic scene imaging and overcoming the difficulty of image registration; the two imaging light beams after splitting enter the high-brightness sensor and low-brightness sensor of the imaging module respectively, and the high-brightness sensor and the low-brightness sensor are synchronously triggered or asynchronously triggered by the variable integral exposure module, and the exposure time of the two sensors is controlled to obtain output images with different exposures.

[0087] In this embodiment, m=10 is specifically used, and the imaging principle and results are as follows: Figure 2 shown.

[0088] This embodiment also provides a high dynamic optical spectroscopic imaging method suitable for dynamic scenes, which controls the range and overlapping area of ​​the sensor's acceptable radiance by means of variable splitting ratio and variable integration time, and can achieve high dynamic range imaging with simple structure and high sensitivity. Figure 3 As shown, the following steps are included:

[0089] S1. Measure the radiance range of the scene using a radiometer, denoted as L min~L max , and collect the scene of the target area through the front lens module to obtain the scene light beam;

[0090] S2, the scene beam is divided into a high-brightness beam and a low-brightness beam with a luminous flux ratio of m:1 through a variable splitter module, and the minimum radiance of the high-brightness sensor and the low-brightness sensor is calculated in combination with the radiance range of the scene obtained in S1; S2 is specifically:

[0091] The scene beam is divided into a high-brightness beam and a low-brightness beam with a luminous flux ratio of m:1 by the variable splitter module, and the minimum radiance L of the high-brightness sensor is calculated by the following formulas in combination with the radiance range of the scene obtained by S1: HE_min , the minimum radiance L of the low-light sensor LE_min :

[0092]

[0093] Among them, L min Indicates the minimum radiance of the scene.

[0094] S3, take the longest integration time when the signal-to-noise ratio (SNR) of the dark part of the scene is greater than 1 as the exposure time t of the highlight sensor HE ; Take the inherent minimum integration time of the low-light sensor as the exposure time t of the low-light sensor LE .

[0095] S4. Calculate the maximum radiance value when the high-brightness sensor and the low-brightness sensor are saturated, and combine the minimum radiance of the high-brightness sensor and the low-brightness sensor to calculate the radiance range of the actual scene captured:

[0096] Calculate the maximum radiance value L when the highlight sensor pixel is saturated HE_max The formula is:

[0097]

[0098] Among them, DN HE_max is the maximum pixel value of the highlight sensor; S HE_max is the number of electrons excited in the highlight sensor; C is the conversion efficiency; A is the photosensitive area of ​​the pixel of the front lens module; ε is the shielding coefficient; F is the ratio of the equivalent focal length and optical aperture of the combination of the front lens module and the variable diffraction module; h is the Planck constant; c is the speed of light; λ is the median wavelength of the incident light; τ is the transmittance of the combination of the front lens module and the variable diffraction module; η qH is the quantum efficiency of the high brightness sensor, t HE is the exposure time of the highlight sensor;

[0099] The maximum radiance value L when the low-light sensor pixel is saturated LE_maxThe calculation formula is:

[0100]

[0101] Among them, DN LE_max is the maximum pixel value of the low-light sensor; S LE_max is the number of electrons excited in the low-light sensor; t LE is the exposure time of the low-light sensor, η qL is the quantum efficiency of the low-light sensor;

[0102] Determine the radiance range actually received by the highlight sensor as:

[0103] △L HE_accept =L HE_min ~L HE_max ;

[0104] Divided by the splitting ratio, the actual scene radiance range captured by the highlight sensor is:

[0105]

[0106] Determine the actual radiance range received by the low-light sensor:

[0107] △L LE_accept =L LE_min ~L LE_max ;

[0108] Divided by the splitting ratio, the actual scene radiance range captured by the low-light sensor is:

[0109]

[0110] At the same time, a virtual imaging optical path is obtained according to the exposure time of the high-brightness sensor and the low-brightness sensor; the virtual imaging optical path includes a high-brightness virtual optical path and a low-brightness virtual optical path.

[0111] S5. Determine whether the radiance range actually received by the high brightness sensor and the low brightness sensor completely covers the radiance range of the scene. The expression is:

[0112]

[0113] Determine whether the actual radiance range captured by the high brightness sensor and the low brightness sensor has an overlapping area; the expression is:

[0114]

[0115] If both conditions are met, the variable integral exposure module controls the high-brightness sensor and the low-brightness sensor to receive the high-brightness light beam and the low-brightness light beam according to the exposure time, and performs exposure imaging to obtain a multi-exposure image sequence of the target area, thereby completing high-dynamic optical spectroscopic imaging of the scene;

[0116] Otherwise, execute S6;

[0117] S6. Extend the exposure time of the low-brightness sensor and use the virtual imaging optical path to compensate for the radiance range actually received by the low-brightness sensor. The specific process is as follows:

[0118] Take the exposure time of the low-light sensor extended for the i-th time as a represents the ratio of the exposure time of the low-light sensor extended for the i-th time to the exposure time of the low-light sensor before the extension;

[0119] The radiance range actually received by the virtual imaging optical path for:

[0120]

[0121] in, represents the minimum radiance of the virtual imaging light path obtained by changing the exposure time for the i-th time, represents the maximum radiance of the virtual imaging light path obtained by changing the exposure time for the i-th time;

[0122] Determine whether the radiance range actually received by the virtual imaging optical path overlaps with the radiance range actually received by the low-brightness sensor. The judgment formula is as follows:

[0123] Is it

[0124] If not There is overlap, and the radiance range actually received by the low-brightness sensor becomes:

[0125]

[0126] If If there is no overlap, return to modify and extend the exposure time of the low-light sensor.

[0127] Calculate the actual scene radiance range captured by the high-brightness sensor and the low-brightness sensor, and determine whether the captured actual scene radiance range can completely cover the scene dynamic range. If so, control the high-brightness sensor and the low-brightness sensor to receive the high-brightness light beam and the low-brightness light beam respectively through the variable integral exposure module according to the currently determined exposure time, and perform exposure imaging to obtain a multi-exposure image sequence of the target area, thereby completing high-dynamic optical spectroscopic imaging of the scene;

[0128] Otherwise, return to S4, adjust the exposure time of the high-brightness sensor, and update the exposure time of the low-brightness sensor to the extended exposure time of the low-brightness sensor, until a multi-exposure image sequence of the target area is obtained, completing the high-dynamic optical spectroscopic imaging of the scene.

[0129] In this embodiment, the exposure time of the sensor is adjusted by a variable integral exposure module. When the number of sensors and the splitting ratio are fixed, a new image acquisition method is added to compensate for the radiance range. Figure 4 A schematic diagram of adding a virtual imaging light path to cover the entire radiance area by changing the low-brightness sensor exposure mode is given; Figure 4 It can be seen that the originally missing radiance interval can be covered by the virtual light path. Each exposure of each light path has an intersection with the adjacent exposure (including redundant information). Therefore, the spectroscopic system can contain the brightness information and detail information of the scene in the multiple exposure image sequence, providing quality assurance for the subsequent HDR image / video reconstruction work.

[0130] In this embodiment, the exposure time t of the highlight sensor is adjusted HE The specific method is: reduce the exposure time of the highlight sensor exponentially.

[0131] In this embodiment, two sensors are used to synchronously image and collect images. The image frame sequence obtained is as follows: Figure 5 As shown, from Figure 5 It can be seen that during the imaging period, the high-brightness sensor and the low-brightness sensor simultaneously receive the imaging beam of the variable splitter module at a fixed frame rate. The high-brightness sensor continuously acquires the high-brightness image with a fixed integration time, and the variable integral exposure module synchronously triggers the low-brightness sensor. When the image acquisition conditions are not met, the exposure time is changed alternately to perform cyclic acquisition of LE1-LE2 images. The virtual light path obtained by alternating exposure can make up for the limitations brought by the splitting ratio and the number of sensors, flexibly realize the imaging requirements of different span dynamic ranges, equivalently realize the performance of the multi-channel splitting imaging system, and avoid the complexity of the multi-channel splitting system, which is conducive to the manufacture of large-aperture imaging systems.

[0132] Example 2

[0133] Different from the first embodiment, the present embodiment adopts a working mode of asynchronous imaging acquisition of two sensors, and asynchronous acquisition is a working mode for realizing high frame rate image acquisition. Figure 6 It can be seen that during the imaging period, the high-brightness sensor and the low-brightness sensor continuously acquire images in the S5 mode to obtain the image sequence required for high dynamic range image reconstruction. The difference from S5 is that the high-brightness sensor and the low-brightness sensor can be asynchronously triggered by the variable integral exposure module, such as Figure 6When the asynchronous triggering time interval between the high brightness sensor and the low brightness sensor is half of the original frame cycle time, the temporal resolution of the imaging system can be doubled.

[0134] In the high frame rate acquisition mode of this embodiment, the image acquisition phase relationship between multiple sensors is utilized to enhance the temporal resolution of the entire sensor without reducing the single-channel exposure time. By introducing a phase offset between these sensors, their outputs can be staggered in time, thereby improving the overall sampling efficiency.

[0135] Comparative Example

[0136] Different from Example 1, this comparative example follows S1 to S3 in a high-dynamic optical spectroscopic imaging method suitable for dynamic scenes, and controls the high-brightness sensor and the low-brightness sensor to receive the high-brightness light beam and the low-brightness light beam according to the exposure time in S3, and performs exposure imaging; in this comparative example, when only the variable spectroscope module is used to image an extremely high-dynamic scene, the splitting ratio is often too extreme, resulting in no overlap in the radiant brightness range obtained by the two sensors, such as Figure 7 shown; from Figure 7 It can be seen that due to the limitation of the number of sensors and the fixed splitting ratio, the imaging system may not be able to fully cover the radiance of the scene when facing extreme scenes, resulting in the non-overlapping radiance ranges that can be captured by the two sensors, leading to the loss of the radiance information of the scene.

[0137] The above description is only a specific embodiment of the present invention, and a comparison of the effects of the specific embodiments and the related comparative examples, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A high dynamic optical spectroscopic imaging system suitable for dynamic scenes, comprising a front lens module in the form of a rear spectroscopic single camera system, characterized in that: Also includes: A variable beam splitter module, an imaging module, and a variable integral exposure module connected to the imaging module; the front lens module, the variable beam splitter module, and the imaging module are arranged in sequence along the optical path; The front lens module is used to adjust the aperture and focal length, and collect scene light beams for the target area; The variable beam splitter module includes a beam splitter prism or a neutral density filter, which is used to split the scene light beam according to the light flux of m:1 into a high-brightness light beam and a low-brightness light beam, and change the transmission direction of the light beam; The imaging module includes a high brightness sensor and a low brightness sensor, which are respectively used to receive a high brightness light beam and a low brightness light beam, perform exposure imaging, and output a multi-exposure image sequence of the target area; The variable integral exposure module includes a data processing controller, which is respectively connected to a high brightness sensor and a low brightness sensor, and is used to control synchronous triggering or asynchronous triggering of the high brightness sensor and the low brightness sensor, and control the exposure time of the high brightness sensor and the low brightness sensor.

2. A high dynamic optical spectroscopic imaging system suitable for dynamic scenes according to claim 1, characterized in that: The value range of m is 1≤m≤20.

3. A high dynamic optical spectroscopic imaging method suitable for dynamic scenes, using a high dynamic optical spectroscopic imaging system suitable for dynamic scenes as claimed in claim 1 or 2, characterized in that: The following steps are involved: S1, measuring the radiance range of the scene, and collecting the scene of the target area through the front lens module to obtain the scene light beam; S2, splitting the scene beam into a high-brightness beam and a low-brightness beam with a luminous flux ratio of m:1 through a variable splitter module, and calculating the minimum radiance of the high-brightness sensor and the low-brightness sensor in combination with the radiance range of the scene obtained in S1; S3. Determine the exposure time t of the highlight sensor HE and the exposure time t of the low-light sensor LE ; S4, calculating the maximum radiance value of the high brightness sensor and the low brightness sensor when the pixels are saturated, and combining the minimum radiance of the high brightness sensor and the low brightness sensor, calculating the radiance range actually received by the high brightness sensor and the low brightness sensor, and the radiance range of the actual scene captured; at the same time, obtaining the virtual imaging optical path according to the exposure time of the high brightness sensor and the low brightness sensor; S5, determining whether the radiance range actually received by the high brightness sensor and the low brightness sensor completely covers the radiance range of the scene and whether there is an overlapping area in the actual radiance range captured by the high brightness sensor and the low brightness sensor; If both conditions are met, the variable integral exposure module controls the high-brightness sensor and the low-brightness sensor to receive the high-brightness light beam and the low-brightness light beam according to the exposure time, and performs exposure imaging to obtain a multi-exposure image sequence of the target area, thereby completing high-dynamic optical spectroscopic imaging of the scene; Otherwise, execute S6; S6. Extend the exposure time of the low-brightness sensor, use the virtual imaging optical path to compensate for the radiance range actually received by the low-brightness sensor, calculate the actual scene radiance range captured by the high-brightness sensor and the low-brightness sensor, and determine whether the captured actual scene radiance range can fully cover the scene dynamic range; If yes, the variable integral exposure module controls the high brightness sensor and the low brightness sensor to receive the high brightness light beam and the low brightness light beam according to the currently determined exposure time, and performs exposure imaging to obtain a multi-exposure image sequence of the target area, thereby completing high dynamic optical spectroscopic imaging of the scene; Otherwise, return to S4, adjust the exposure time of the high-brightness sensor, and update the exposure time of the low-brightness sensor to the extended exposure time of the low-brightness sensor, until a multi-exposure image sequence of the target area is obtained, completing the high-dynamic optical spectroscopic imaging of the scene.

4. The high dynamic optical spectroscopic imaging method suitable for dynamic scenes according to claim 3, characterized in that: S2 is specifically: The scene beam is divided into a high-brightness beam and a low-brightness beam with a luminous flux ratio of m:1 by the variable splitter module, and the minimum radiance L of the high-brightness sensor is calculated by the following formulas in combination with the radiance range of the scene obtained by S1: HE_min , the minimum radiance L of the low-light sensor LE_min : Among them, L min Indicates the minimum radiance of the scene.

5. The high dynamic optical spectroscopic imaging method suitable for dynamic scenes according to claim 3, characterized in that: The exposure time of the high brightness sensor and the low brightness sensor is determined in S3 as follows: the longest integration time when the signal-to-noise ratio SNR of the dark part signal of the scene is greater than 1 is taken as the exposure time t of the high brightness sensor. HE ; Take the inherent minimum integration time of the low-light sensor as the exposure time t of the low-light sensor LE .

6. The high dynamic optical spectroscopic imaging method suitable for dynamic scenes according to claim 4, characterized in that: The maximum radiance value when the high-brightness sensor and the low-brightness sensor are saturated is calculated in S4, and combined with the minimum radiance of the high-brightness sensor and the low-brightness sensor, the radiance range of the actual scene captured is calculated as follows: Calculate the maximum radiance value L when the highlight sensor pixel is saturated HE_max The formula is: Among them, DN HE_max is the maximum pixel value of the highlight sensor; S HE_max is the number of electrons excited in the highlight sensor; C is the conversion efficiency; A is the photosensitive area of ​​the pixel of the front lens module; ε is the shielding coefficient; F is the ratio of the equivalent focal length and optical aperture of the combination of the front lens module and the variable diffraction module; h is the Planck constant; c is the speed of light; λ is the median wavelength of the incident light; τ is the transmittance of the combination of the front lens module and the variable diffraction module; η qH is the quantum efficiency of the high brightness sensor, t HE is the exposure time of the highlight sensor; The maximum radiance value L when the low-light sensor pixel is saturated LE_max The calculation formula is: Among them, DN LE_max is the maximum pixel value of the low-light sensor; S LE_max is the number of electrons excited in the low-light sensor; t LE is the exposure time of the low-light sensor, η qL is the quantum efficiency of the low-light sensor; Determine the radiance range actually received by the highlight sensor as: △L HE_accept =L HE_min ~L HE_max ; Divided by the splitting ratio, the actual scene radiance range captured by the highlight sensor is: Determine the actual radiance range received by the low-light sensor: △L LE_accept =L LE_min ~L LE_max ; Divided by the splitting ratio, the actual scene radiance range captured by the low-light sensor is:

7. A high dynamic optical spectroscopic imaging method suitable for dynamic scenes according to claim 6, characterized in that: In S5, the expression for judging whether the radiance range actually received by the high brightness sensor and the low brightness sensor completely covers the radiance range of the scene is: And L LE_max -L HE_min >0; The expression for judging whether the actual radiance range of the high brightness sensor and the low brightness sensor has an overlapping area is:

8. The high dynamic optical spectroscopic imaging method suitable for dynamic scenes according to claim 6, characterized in that: The specific process of extending the exposure time of the low-brightness sensor in S6 and using the virtual imaging optical path to compensate for the radiance range actually received by the low-brightness sensor is as follows: Take the exposure time of the low-light sensor extended for the i-th time as a represents the ratio of the exposure time of the low-light sensor extended for the i-th time to the exposure time of the low-light sensor before the extension, where a≥1.1; The radiance range actually received by the virtual imaging optical path for: in, represents the minimum radiance of the virtual imaging light path obtained by changing the exposure time for the i-th time, represents the maximum radiance of the virtual imaging light path obtained by changing the exposure time for the i-th time; Determine whether the radiance range actually received by the virtual imaging optical path overlaps with the radiance range actually received by the low-brightness sensor. The judgment formula is as follows: Is it If not There is overlap, and the radiance range actually received by the low-brightness sensor becomes: If If there is no overlap, return to modify and extend the exposure time of the low-light sensor.

9. The high dynamic optical spectroscopic imaging method suitable for dynamic scenes according to claim 3, characterized in that: Adjust the exposure time t of the highlight sensor as described in S6 HE Specifically: Reduce the exposure time of the high-brightness sensor exponentially.

10. The high dynamic optical spectroscopic imaging method suitable for dynamic scenes according to claim 3, characterized in that: The instrument for measuring the radiance range of a scene described in S1 is a radiometer.

Images