A high dynamic optical spectral imaging system and imaging method suitable for dynamic scenes

Abstract

The application discloses a high dynamic optical light splitting imaging system and imaging method suitable for a dynamic scene, belongs to an optoelectronic sensor imaging system, and solves the problem of complex operation of a high dynamic imaging system, and comprises a front lens module, a variable ratio light splitter module, a variable integral exposure module and an imaging module; wherein the front lens module is used for adjusting an aperture and a focal length, the imaging system as a whole adopts a rear light splitting single camera system form, a set of front imaging optical machine system is used for the main body to adapt to the limitation of a large aperture camera on weight, cost and structural complexity; the imaging module adopts two-way sensors to simultaneously image a motion scene, and registration and image artifacts caused by multi-frame imaging are avoided; and a flexible acquisition strategy is introduced in the imaging method, and each sensor is exposed through a variable integral time to make the capturable brightness intervals of the sensors overlap, so that the expansion of the dynamic range of the system is completed.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of photoelectric sensor imaging system, and particularly relates to a high dynamic optical spectral imaging system and imaging method suitable for dynamic scenes. BACKGROUND

[0002] Under high-contrast lighting conditions, the dynamic range of a scene often does not match the dynamic range that can be recorded by an imaging system, which brings difficulties to post-image processing and scene recognition. In order to solve this problem, high dynamic imaging technology has been researched for many years to overcome the contradiction between the high dynamic range (HDR) characteristics of a real scene and the low dynamic range (LDR) limitations of a camera. At present, high dynamic optical imaging technology mainly has the following three forms:

[0003] (1) High dynamic imaging based on image sensor hardware, which mainly focuses on directly improving the dynamic range performance of a camera image sensor through measures such as reducing noise, improving quantum efficiency, and improving full-well capacity. This form can inherit the existing imaging optical-mechanical structure, is directly installed, and is easy to use, but is limited by factors such as device materials, design level, semiconductor production equipment and process level, and is expensive.

[0004] (2) High dynamic imaging based on multiple exposures of a single camera, which mainly uses a camera to continuously take a sequence of images with different exposures, uses the brightness mapping between the images, and fuses the effective areas of the images with different exposures to obtain a high dynamic image. This method has low requirements for equipment, but requires the scene to remain stable and be captured within a short time, so its application effect in dynamic scenes is poor.

[0005] (3) High dynamic imaging based on multiple image sensors, which mainly includes a multi-view camera array system, a front-splitting multi-camera system, and a rear-splitting single-camera system. The multi-view camera array system uses multiple cameras to simultaneously image a scene, which has the disadvantage of parallax artifacts caused by spatial position differences. The front-splitting multi-camera system has a splitting device at the front end of the entire system, and multiple cameras receive different branch light energies, which has the disadvantages of complex system structure, high cost, and unsuitability for large-aperture imaging systems. The rear-splitting single-camera system uses a front optical-mechanical structure, and a splitting device is located at the rear end of the system, and multiple image sensors receive different branch light energies, which has the advantages of relatively simple system structure, low cost, suitability for dynamic scene imaging, and suitability for manufacturing large-aperture imaging systems. However, the existing rear-splitting single-camera system mostly adopts a single light path form, which is prone to problems such as incomplete coverage of the dynamic range of a scene, loss of scene information, and inability to recover scene information.

[0006] Through the analysis of the above prior art status, it is known that in current imaging applications, the scene dynamic range is often too large, and the imaging device is difficult to present all the information in the scene in an image. Single camera multiple exposure imaging sacrifices imaging real-time to expand the dynamic range, which is prone to artifacts caused by late registration, and is difficult to adapt to the case of scene target motion and view angle change. The hardware scheme of high dynamic image sensor is limited by the existing research level, technical barriers and expensive price, and the multi-view camera array system and the front split light multi-camera system face the problems of system complexity, high manufacturing cost and difficulty in adapting to large aperture imaging system. SUMMARY

[0007] In order to solve the problems that the hardware scheme of high dynamic image sensor is limited by the existing research level and expensive price, and the multi-view camera array system and the front split light multi-camera system face the problems of system complexity, high manufacturing cost and difficulty in adapting to large aperture imaging system, and the rear split light single camera system is prone to scene information loss and scene information recovery problems caused by incomplete coverage of scene dynamic range, the application provides a high dynamic optical split imaging system and imaging method suitable for dynamic scenes.

[0008] The application concept of the application is to abandon the complex multi-camera system form, and to adopt a system form of integrating rear two-way split light in a single lens camera, which can avoid the problems of large increase in system structure complexity and high cost, and can be applied to the manufacture of large aperture imaging system.

[0009] In order to achieve the above purpose and complete the above application concept, the application adopts the following technical scheme:

[0010] A high dynamic optical split imaging system suitable for dynamic scenes, comprising a front lens module in the form of a rear split light single camera system, characterized in that it further comprises:

[0011] a variable ratio splitter module, an imaging module, and a variable integral exposure module connected with the imaging module; the front lens module, the variable ratio splitter module and the imaging module are arranged in sequence along the optical path;

[0012] The front lens module is used to adjust the aperture and focal length, and collect scene light beams of the target area; a set of front imaging optical system is used for the main body to adapt to the limitations of weight, cost and structural complexity of large aperture camera;

[0013] The variable ratio splitter module comprises a split prism or a neutral density filter, which is used to split the scene light beams according to the light flux of m:1, into high brightness light beams and low brightness light beams, and change the transmission direction of the light beams, so as to realize double optical path imaging of the same scene and the same view angle, meet the needs of dynamic scene imaging, overcome the difficulty of image registration, and obtain two imaging light beams;

[0014] The imaging module comprises a high-exposure sensor and a low-exposure sensor, the image sensor with a larger light exposure is referred to as a high-exposure (HE) sensor, the imaging result of which is overall bright, and the dark environment in the scene can be prevented from being affected by noise; the image sensor with a smaller light exposure is referred to as a low-exposure (LE) sensor, the imaging result of which is overall dark, and the overexposure phenomenon in a high-exposure scene can be inhibited, the high-exposure sensor and the low-exposure sensor are respectively used for receiving a high-exposure light beam and a low-exposure light beam and performing exposure imaging to output a multi-exposure image sequence of a target region; through simultaneous imaging of the high-exposure sensor and the low-exposure sensor, registration and image artifacts caused by multi-frame imaging can be avoided.

[0015] The variable integral exposure module comprises a data processing controller connected with the high-exposure sensor and the low-exposure sensor respectively, and is used for controlling synchronous triggering or asynchronous triggering of the high-exposure sensor and the low-exposure sensor and controlling exposure times of the high-exposure sensor and the low-exposure sensor.

[0016] The high-dynamic optical split imaging system suitable for a dynamic scene discards a complex multi-camera system form, selects a system form of integrating two rear split light beams in a single lens camera, avoids a problem of a large increase in system structure complexity and excessively high cost, and is suitable for manufacturing of a large-aperture imaging system.

[0017] The value range of m is 1<=m<=20.

[0018] Meanwhile, the application further provides a high-dynamic optical split imaging method suitable for a dynamic scene, which uses the high-dynamic optical split imaging system suitable for a dynamic scene, and the speciality thereof lies in comprising the following steps:

[0019] S1, measuring a radiance range of a scene, denoted as L min ~L max , and collecting a scene light beam of the target region through the front lens module;

[0020] S2, splitting the scene light beam into a high-exposure light beam and a low-exposure light beam with a light flux ratio of m:1 through the variable ratio light splitter module, and calculating minimum radiance of the high-exposure sensor and the low-exposure sensor in combination with the radiance range of the scene obtained in S1;

[0021] S3, determining an exposure time t HE of the high-exposure sensor and an exposure time t LE of the low-exposure sensor;

[0022] S4, calculate the maximum radiance value when the high-light sensor and low-light sensor pixels are saturated, and combine the minimum radiance of the high-light sensor and low-light sensor to calculate the actual received radiance range of the high-light sensor and low-light sensor, the actual scene radiance range captured; at the same time, obtain the virtual imaging light path according to the exposure time of the high-light sensor and low-light sensor;

[0023] S5, judge whether the actual received radiance range of the high-light sensor and low-light sensor completely covers the radiance range of the scene and whether there is an overlapping area in the actual radiance range captured by the high-light sensor and low-light sensor;

[0024] If both conditions are met, then the high-light sensor and low-light sensor receive high-light beams and low-light beams according to the exposure time controlled by the variable integral exposure module, and exposure imaging is performed to obtain a multi-exposure image sequence of the target region, and high dynamic optical spectral imaging of the scene is completed;

[0025] Otherwise, S6 is executed;

[0026] S6, extend the exposure time of the low-light sensor, compensate for the actual received radiance range of the low-light sensor using the virtual imaging light path, calculate the actual scene radiance range captured by the high-light sensor and low-light sensor, and judge whether the actual scene radiance range captured can completely cover the dynamic range of the scene;

[0027] If so, the high-light sensor and low-light sensor receive high-light beams and low-light beams, respectively, according to the exposure time determined at present, and exposure imaging is performed to obtain a multi-exposure image sequence of the target region, and high dynamic optical spectral imaging of the scene is completed;

[0028] Otherwise, return to S4, adjust the exposure time of the high-light sensor, and update the exposure time of the low-light sensor in it to the extended exposure time of the low-light sensor, until a multi-exposure image sequence of the target region is obtained, and high dynamic optical spectral imaging of the scene is completed.

[0029] Further, S2 is specifically:

[0030] The scene light beam is divided into high-light beams and low-light beams with a light flux ratio of m:1 by the variable ratio beam splitter module, and the minimum radiance of the high-light sensor L HE_min and the minimum radiance of the low-light sensor L LE_min are calculated by the following formulas, respectively, according to the scene radiance range obtained in S1:

[0031]

[0032] wherein, L min represents the minimum value of the scene radiance.

[0033] Further, the specific process of determining the exposure time of the highlight sensor and the low-light sensor in S3 is: taking the longest integration time of the scene dark part signal with the signal-to-noise ratio SNR>1 as the exposure time t HE of the highlight sensor LE .

[0034] Further, the specific process of calculating the maximum radiance value when the pixels of the highlight sensor and the low-light sensor are saturated in S4 is: calculating the minimum radiance of the highlight sensor and the low-light sensor, and combining the minimum radiance of the highlight sensor and the low-light sensor to calculate the actual scene radiance range captured.

[0035] The maximum radiance value L HE_max when the pixels of the highlight sensor are saturated is calculated according to the formula:

[0036]

[0037] DN HE_max is the maximum pixel value of the highlight sensor; S HE_max is the number of excited electrons in the highlight sensor; C is the conversion efficiency; A is the pixel photosensitive area of the front lens module; ε is the shielding coefficient; F is the ratio of the equivalent focal length and the optical aperture of the combination of the front lens module and the variable ratio beam splitter module; h is the Planck constant; c is the speed of light; λ is the median wavelength of the incident light; τ is the transmittance of the combination of the front lens module and the variable ratio beam splitter module; η qH is the quantum efficiency of the highlight sensor, and t HE is the exposure time of the highlight sensor.

[0038] The maximum radiance value L LE_max when the pixels of the low-light sensor are saturated is calculated according to the formula:

[0039]

[0040] DN LE_max is the maximum pixel value of the low-light sensor; S LE_max is the number of excited electrons in the low-light sensor; t LE is the exposure time of the low-light sensor, and η qL is the quantum efficiency of the low-light sensor.

[0041] The actual received radiance range of the highlight sensor is determined as:

[0042] △L HE_accept = L HE_min ~ L HE_max ;

[0043] Divided by the beam splitting ratio, the actual scene radiance range captured by the highlight sensor is:

[0044]

[0045] The actual received radiance range of the low-light sensor is △L LE_accept = L LE_min ~ L LE_max ;

[0046] Divided by the spectral ratio, the actual scene radiance range captured by the low-light sensor is:

[0047]

[0048] Further, the virtual imaging light path in S4 includes a high-light virtual light path and a low-light virtual light path.

[0049] Further, the expression for determining whether the actual received radiance range of the high-light sensor and the low-light sensor completely covers the radiance range of the scene in S5 is:

[0050]

[0051] The expression for determining whether the actual radiance range of the high-light sensor and the low-light sensor has an overlapping area is:

[0052]

[0053] Further, the specific process for extending the exposure time of the low-light sensor and compensating for the actual received radiance range of the low-light sensor using the virtual imaging light path in S6 is:

[0054] The i-th extended exposure time of the low-light sensor is taken as T a represents the ratio of the i-th extended exposure time of the low-light sensor to the exposure time of the low-light sensor before extension, where a≥1.1;

[0055] The actual received radiance range of the virtual imaging light path is:

[0056]

[0057] Wherein, represents the minimum radiance of the virtual imaging light path obtained by changing the exposure time for the i-th time, represents the maximum radiance of the virtual imaging light path obtained by changing the exposure time for the i-th time;

[0058] The judgment formula for determining whether the actual received radiance range of the virtual imaging light path overlaps with the actual received radiance range of the low-light sensor is:

[0059] is​

[0060] If not If there is overlap, the actual received radiance range of the low-brightness sensor becomes:

[0061]

[0062] If If there is no overlap, return to modify the exposure time of the low-brightness sensor.

[0063] Further, the adjustment of the exposure time t of the high-brightness sensor in S6 is: HE Specifically: multiply the exposure time of the high-brightness sensor.

[0064] Further, the instrument for measuring the radiance range of the scene in S1 is a radiance meter.

[0065] The beneficial effects of the present application are:

[0066] 1. The high-dynamic optical splitting imaging system suitable for dynamic scenes of the present application discards the complex multi-camera system form, selects the system form of integrating the post-positioned two-way splitting in the front lens module, avoids the problems of large increase in system structure complexity and high cost, and is suitable for manufacturing of large-aperture imaging systems.

[0067] 2. In the high-dynamic optical splitting imaging system suitable for dynamic scenes of the present application, high-brightness sensors and low-brightness sensors are used to simultaneously image the moving scene, avoiding registration and image artifacts caused by multi-frame imaging.

[0068] 3. In the high-dynamic optical splitting imaging method suitable for dynamic scenes of the present application, the actual obtainable scene radiance range of each sensor is controlled by adjusting the splitting ratio, and the expansion of the primary dynamic range is realized through the "optical lever" effect.

[0069] 4. In the high-dynamic optical splitting imaging method suitable for dynamic scenes of the present application, a flexible acquisition strategy is introduced, and each sensor is exposed by varying the integration time to make the captureable brightness intervals of each sensor overlap, thereby completing the expansion of the secondary dynamic range of the system and preventing the loss of scene information.

[0070] 5. In the high-dynamic optical splitting imaging method suitable for dynamic scenes of the present application, the two-way sensors can be used in an asynchronous acquisition manner to multiply the system frame frequency, thereby improving the sampling efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0071] Figure 1 is a structural schematic view of embodiment 1 of the high-dynamic optical splitting imaging system suitable for dynamic scenes of the present application.

[0072] Figure 2 This is a schematic diagram illustrating the imaging principle and results of Embodiment 1 of a high dynamic optical beam splitting imaging system suitable for dynamic scenes according to the present invention;

[0073] Figure 3 This is a flowchart of S1 to S5 in Embodiment 1 of the High Dynamic Range Optical Spectroscopic Imaging Method for Dynamic Scenes of the present invention;

[0074] Figure 4 This is a schematic diagram of adding a virtual imaging optical path to supplement the missing radiance range in S6 of Embodiment 1 of the High Dynamic Range Optical Spectroscopic Imaging Method for Dynamic Scenes of the present invention.

[0075] Figure 5 This is a schematic diagram of the image frame sequence obtained in the synchronous acquisition mode of Embodiment 1 of the high dynamic optical beam splitting imaging method suitable for dynamic scenes of the present invention;

[0076] Figure 6 This is the asynchronous acquisition mode in Embodiment 2 of the high dynamic optical beam splitting imaging method suitable for dynamic scenes of the present invention;

[0077] Figure 7 This is a schematic diagram illustrating the loss of radiance region caused by using the comparative example of the present invention without adjusting the integration time. Detailed Implementation

[0078] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0079] Example 1

[0080] This invention provides a high dynamic range optical beam splitting imaging system suitable for dynamic scenes, such as... Figure 1 As shown, it includes:

[0081] The front-facing lens module adopts a rear-mounted beam splitter single-camera system to adjust the aperture and focal length, and to capture scene beams in the target area; the main body uses a front-facing imaging optical engine system to adapt to the weight, cost and structural complexity limitations of large-aperture cameras.

[0082] The variable beam splitter module includes a beam splitter prism or neutral density filter, which is used to split the scene beam into a high-brightness beam and a low-brightness beam according to the luminous flux ratio of m:1 (1≤m≤20), and change the beam transmission direction to realize dual-optical-path imaging of the same scene from the same perspective, meet the needs of dynamic scene imaging, and overcome the difficulties of image registration.

[0083] The imaging module includes a high-exposure sensor and a low-exposure sensor. The image sensor with a larger light exposure is referred to as a high-exposure (HE) sensor, and the imaging result of the high-exposure sensor is overall bright, which can avoid the influence of noise on dark environments in the scene. The image sensor with a smaller light exposure is referred to as a low-exposure (LE) sensor, and the imaging result of the low-exposure sensor is overall dark, which can suppress the overexposure phenomenon in a high-exposure scene. The high-exposure sensor and the low-exposure sensor are respectively used to receive a high-exposure light beam and a low-exposure light beam and perform exposure imaging to output a multi-exposure image sequence of a target region. In this embodiment, the imaging module is connected to a storage device, and the multi-exposure image sequence of the target region output by the imaging module can be stored to facilitate the synthesis of the multi-exposure image sequence of the target region.

[0084] In this embodiment, the high-exposure sensor and the low-exposure sensor simultaneously perform imaging, which can avoid registration and image artifacts caused by multi-frame imaging.

[0085] The variable integral exposure module includes a data processing controller connected to the high-exposure sensor and the low-exposure sensor, respectively, and is used to control the synchronous triggering or asynchronous triggering of the high-exposure sensor and the low-exposure sensor and control the exposure time of the high-exposure sensor and the low-exposure sensor.

[0086] In this embodiment, the imaging system overall adopts a rear-mounted split light single-camera system form, and a front-mounted imaging optical system is used in the main body to adapt to the limitations of large-aperture cameras on weight, cost, and structural complexity. The scene light beam of the target region is input into the variable-ratio light splitter module, the light flux is split at a split ratio of m:1 (m>1), the transmission direction of the light beam is changed, two imaging light beams are obtained, and the two imaging light beams are split into two beams. The same scene and the same angle of view can be imaged by the two light paths, the need for dynamic scene imaging is met, and the difficulty of image registration is overcome. The two imaging light beams after splitting enter the high-exposure sensor and the low-exposure sensor of the imaging module, respectively. The variable integral exposure module synchronously triggers or asynchronously triggers the high-exposure sensor and the low-exposure sensor, and controls the exposure time of the two sensors to obtain output images with different exposure degrees.

[0087] In this embodiment, m=10 is specifically used, and the imaging principle and result are as shown in Figure 2 .

[0088] In this embodiment, a high-dynamic optical split imaging method suitable for dynamic scenes is also provided. The variable split ratio and the variable integral time are used to control the range and overlapping area of the sensor acceptable radiant brightness, and the high-dynamic range imaging with simple structure and high sensitivity can be realized. S1-S5 are as shown in Figure 3 .

[0089] S1, measure the radiant brightness range of the scene by a radiant brightness meter, and record the radiant brightness range as L min~L max And the scene of the target area is collected by the front lens module to obtain a scene light beam;

[0090] S2, the scene light beam is divided into a high-brightness light beam and a low-brightness light beam with a light flux ratio of m:1 through a variable ratio beam splitter module, and the minimum radiance of the high-brightness sensor and the low-brightness sensor is calculated in combination with the radiance range of the scene obtained in S1; S2 is specifically:

[0091] The scene light beam is divided into a high-brightness light beam and a low-brightness light beam with a light flux ratio of m:1 through a variable ratio beam splitter module, and the minimum radiance of the high-brightness sensor and the low-brightness sensor is calculated in combination with the radiance range of the scene obtained in S1; S2 is specifically: HE_min LE_min

[0092]

[0093] Wherein, L min represents the minimum value of the radiance of the scene.

[0094] S3, the longest integration time of the scene dark part signal with a signal-to-noise ratio SNR>1 is taken as the exposure time t HE of the high-brightness sensor; and the inherent minimum integration time of the low-brightness sensor is taken as the exposure time t LE of the low-brightness sensor.

[0095] S4, the maximum radiance value when the high-brightness sensor and the low-brightness sensor pixels are saturated is calculated, and the actual scene radiance range of the capture is calculated in combination with the minimum radiance of the high-brightness sensor and the low-brightness sensor:

[0096] The maximum radiance value L HE_max when the high-brightness sensor pixels are saturated is calculated, and the formula is:

[0097]

[0098] Wherein, DN HE_max is the maximum pixel value of the high-brightness sensor; S HE_max is the number of electrons excited in the high-brightness sensor; C is the conversion efficiency; A is the pixel photosensitive area of the front lens module; ε is the shielding coefficient; F is the ratio of the equivalent focal length to the optical aperture of the combination of the front lens module and the variable ratio beam splitter module; h is the Planck constant; c is the light speed; λ is the median wavelength of the incident light; τ is the transmittance of the combination of the front lens module and the variable ratio beam splitter module; η qH is the quantum efficiency of the high-brightness sensor, and t HE is the exposure time of the high-brightness sensor.

[0099] The maximum radiance value L LE_max ​​The calculation formula is:

[0100]

[0101] DN = S * t * η LE_max is the maximum pixel value of the low-light sensor; S LE_max is the number of excited electrons in the low-light sensor; t LE is the exposure time of the low-light sensor, η qL is the quantum efficiency of the low-light sensor;

[0102] The actual received radiance range of the high-light sensor is determined as:

[0103] △L HE_accept = L HE_min ~ L HE_max ;

[0104] Divided by the spectral ratio, the actual scene radiance range captured by the high-light sensor is:

[0105]

[0106] The actual received radiance range of the low-light sensor is determined as:

[0107] △L LE_accept = L LE_min ~ L LE_max ;

[0108] Divided by the spectral ratio, the actual scene radiance range captured by the low-light sensor is:

[0109]

[0110] Meanwhile, the virtual imaging light path is obtained according to the exposure time of the high-light sensor and the low-light sensor. The virtual imaging light path includes a high-light virtual light path and a low-light virtual light path.

[0111] S5, judge whether the actual received radiance range of the high-light sensor and the low-light sensor completely covers the radiance range of the scene, expressed as:

[0112]

[0113] Judge whether the actual radiance range captured by the high-light sensor and the low-light sensor has an overlapping area; the expression is:

[0114]

[0115] If both are satisfied, the high-brightness sensor and the low-brightness sensor receive high-brightness light beams and low-brightness light beams under the control of the variable integral exposure module according to the exposure time, and exposure imaging is performed to obtain a multi-exposure image sequence of the target region, and high-dynamic optical spectral imaging of the scene is completed.

[0116] Otherwise, S6 is performed.

[0117] S6, the exposure time of the low-brightness sensor is extended, and a virtual imaging light path is used to compensate for the actual received radiance range of the low-brightness sensor. The specific process is as follows:

[0118] The exposure time of the low-brightness sensor is extended for the ith time a represents the ratio of the exposure time of the low-brightness sensor extended for the ith time to the exposure time of the low-brightness sensor before extension.

[0119] The actual received radiance range of the virtual imaging light path is .

[0120]

[0121] Wherein, represents the minimum radiance of the virtual imaging light path obtained by changing the exposure time for the ith time, represents the maximum radiance of the virtual imaging light path obtained by changing the exposure time for the ith time.

[0122] It is judged whether the actual received radiance range of the virtual imaging light path overlaps with the actual received radiance range of the low-brightness sensor. The judgment formula is as follows:

[0123] Whether

[0124] If not , there is an overlap, and the actual received radiance range of the low-brightness sensor becomes

[0125]

[0126] If , there is no overlap, and the exposure time of the low-brightness sensor is returned to be modified.

[0127] The actual scene radiance range captured by the high-brightness sensor and the low-brightness sensor is calculated, and it is judged whether the captured actual scene radiance range can completely cover the scene dynamic range. If so, the high-brightness sensor and the low-brightness sensor receive high-brightness light beams and low-brightness light beams under the control of the variable integral exposure module according to the currently determined exposure time, and exposure imaging is performed to obtain a multi-exposure image sequence of the target region, and high-dynamic optical spectral imaging of the scene is completed.

[0128] Otherwise, return to S4, adjust the exposure time of the highlight sensor, and update the exposure time of the low-light sensor to the prolonged exposure time of the low-light sensor until the multi-exposure image sequence of the target region is obtained, and the high dynamic optical spectral imaging of the scene is completed.

[0129] In this embodiment, the exposure time of the sensor is adjusted by the variable integral exposure module. In the case of fixed sensor number and spectral splitting ratio, a new image acquisition mode is added to compensate for the radiance range, Figure 4 The schematic diagram of adding a virtual imaging light path to cover the entire radiance range by changing the exposure mode of the low-light sensor is given; according to Figure 4 It can be seen that the originally missing radiance interval can be covered by the virtual light path. Each exposure of each light path has an intersection (including redundant information) with the adjacent exposure. Therefore, the spectral splitting system can contain the brightness information and detail information of the scene in the multi-exposure image sequence, which provides quality assurance for subsequent HDR image / video reconstruction work.

[0130] In this embodiment, the exposure time t HE The specific method is to reduce the exposure time of the highlight sensor by a factor.

[0131] In this embodiment, the working mode of synchronous imaging acquisition of two sensors is adopted, and the image frame sequence obtained is as shown in Figure 5 As shown in Figure 5 It can be seen that during imaging, the highlight sensor and the low-light sensor simultaneously receive the imaging light beam of the variable ratio spectral splitter module at a fixed frame rate, the highlight sensor continuously acquires high-light images at a fixed integration time, and the variable integral exposure module synchronously triggers the low-light sensor. When the image acquisition condition is not met, the exposure time is alternately changed, and the LE1-LE2 image is cyclically acquired. The virtual light path obtained by the alternate exposure can compensate for the limitations brought by the spectral splitting ratio and the number of sensors, flexibly realize the imaging demand of different span dynamic range, equivalently realize the performance of the multi-path spectral imaging system, and avoid the complexity of the multi-path spectral imaging system, which is conducive to the manufacture of large-aperture imaging systems.

[0132] Embodiment 2

[0133] Different from embodiment 1, in this embodiment, the working mode of asynchronous imaging acquisition of two sensors is adopted, and the asynchronous acquisition is a working mode to realize high frame rate image acquisition. As shown in Figure 6 It can be seen that during imaging, the highlight sensor and the low-light sensor continuously acquire images in the manner of S5 to obtain the image sequence required for high dynamic range image reconstruction. Different from S5, the highlight sensor and the low-light sensor can be asynchronously triggered by the variable integral exposure module, as shown in Figure 6When the asynchronous triggering time interval between the high-brightness sensor and the low-brightness sensor is half of the original frame period time, the temporal resolution of the imaging system can be doubled.

[0134] In this embodiment, the high frame rate acquisition mode utilizes the image acquisition phase relationship between multiple sensors to enhance the overall temporal resolution of the sensor without reducing the single-channel exposure time. By introducing phase shift between these sensors, their outputs are staggered in time, thereby improving the overall sampling efficiency.

[0135] Comparative Example

[0136] Unlike Example 1, this comparative example follows steps S1 to S3 of a high dynamic range optical beam splitting imaging method suitable for dynamic scenes. In step S3, the exposure time is controlled to allow the high-brightness sensor and low-brightness sensor to receive the high-brightness beam and low-brightness beam, respectively, and then exposure imaging is performed. In this comparative example, using only a variable beam splitter module to image extreme high dynamic range scenes often results in an excessively extreme beam splitting ratio, causing a lack of overlap in the radiance ranges acquired by the two sensors. Figure 7 As shown; from Figure 7 As can be seen, due to the limitations of the number of sensors and the fixed splitting ratio, the imaging system may not be able to fully cover the radiance of the scene when facing extreme scenarios, resulting in the radiance ranges that the two sensors can capture not overlapping, leading to the loss of radiance information of the scene.

[0137] The above description is merely a specific embodiment of the present invention and a comparison of the effects of the specific embodiments with relevant comparative examples. However, the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A high dynamic range optical ptychographic imaging method suitable for dynamic scenes, using a high dynamic range optical ptychographic imaging system suitable for dynamic scenes, said high dynamic range optical ptychographic imaging system comprising a front lens module in the form of a back-illuminated ptychographic single-camera system, characterized in that, The high-dynamic optical spectral imaging system further comprises: a variable-ratio beam splitter module, an imaging module, and a variable integral exposure module connected with the imaging module; the front lens module, the variable-ratio beam splitter module, and the imaging module are sequentially arranged along an optical path; the front lens module is used for adjusting an aperture and a focal length, and collecting scene light beams of a target region; the variable-ratio beam splitter module comprises a beam splitting prism or a neutral density filter, and is used for splitting the scene light beams according to an optical flux ratio of m:1 into high-brightness light beams and low-brightness light beams, and changing a light beam transmission direction, where m>1; the imaging module comprises a high-brightness sensor and a low-brightness sensor, and is used for receiving the high-brightness light beams and the low-brightness light beams respectively, and performing exposure imaging, and outputting a multi-exposure image sequence of the target region; the variable integral exposure module comprises a data processing controller, and is connected with the high-brightness sensor and the low-brightness sensor respectively, and is used for controlling synchronous triggering or asynchronous triggering of the high-brightness sensor and the low-brightness sensor, and controlling exposure times of the high-brightness sensor and the low-brightness sensor; comprises the following steps: S1, measuring a radiance range of a scene, and collecting a scene of a target region through the front lens module to obtain scene light beams; S2, splitting the scene light beams into high-brightness light beams and low-brightness light beams with an optical flux ratio of m:1 through the variable-ratio beam splitter module, and calculating minimum radiance of the high-brightness sensor and the low-brightness sensor in combination with the radiance range of the scene obtained in S1; S3, determining exposure times of the high-brightness sensor and the low-brightness sensor; S4, calculating maximum radiance values when pixels of the high-brightness sensor and the low-brightness sensor are saturated, and calculating an actual received radiance range of the high-brightness sensor and the low-brightness sensor, an actual captured scene radiance range of the high-brightness sensor and the low-brightness sensor in combination with the minimum radiance of the high-brightness sensor and the low-brightness sensor, and obtaining a virtual imaging light path according to the exposure times of the high-brightness sensor and the low-brightness sensor; S5, judging whether the actual received radiance range of the high-brightness sensor and the low-brightness sensor completely covers the radiance range of the scene, and whether there is an overlapping area in the actual captured radiance range of the high-brightness sensor and the low-brightness sensor; if both conditions are met, the high-brightness sensor and the low-brightness sensor receive the high-brightness light beams and the low-brightness light beams through the variable integral exposure module according to the exposure times, and exposure imaging is performed to obtain a multi-exposure image sequence of the target region, and high-dynamic optical spectral imaging of the scene is completed; otherwise, S6 is performed; S6, extending the exposure time of the low-brightness sensor, compensating the actual received radiance range of the low-brightness sensor by using the virtual imaging light path, calculating the actual captured scene radiance range of the high-brightness sensor and the low-brightness sensor, and judging whether the actual captured scene radiance range can completely cover the scene dynamic range; if yes, the high-brightness sensor and the low-brightness sensor receive the high-brightness light beams and the low-brightness light beams through the variable integral exposure module according to the currently determined exposure times, and exposure imaging is performed to obtain a multi-exposure image sequence of the target region, and high-dynamic optical spectral imaging of the scene is completed; Otherwise, return to S4, adjust the exposure time of the highlight sensor, and update the exposure time of the low-light sensor to the prolonged exposure time of the low-light sensor until the multi-exposure image sequence of the target area is obtained, and the high dynamic optical spectral imaging of the scene is completed.

2. The method of claim 1, wherein the method is suitable for dynamic scenes. The value range of m is 1 < m ≤ 20.

3. The method of claim 1, wherein the method is suitable for dynamic scenes. S2 is specifically: The scene light beam is divided into a high-brightness light beam and a low-brightness light beam with a light flux ratio of m:1 by a variable-ratio beam splitter module, and the minimum radiance of the high-brightness sensor and the minimum radiance of the low-brightness sensor are calculated by the following formulas, respectively, in combination with the radiance range of the scene obtained by S1 :​ ； ； wherein represents the minimum value of the luminance of the scene.

4. The method of claim 1, wherein the high dynamic range optical spectral imaging method is suitable for dynamic scenes. The exposure time of the highlight sensor and the low-light sensor determined in S3 is specifically: taking the longest integration time of the signal-to-noise ratio of the dark part of the scene as the exposure time of the highlight sensor ; and taking the inherent minimum integration time of the low-light sensor as the exposure time of the low-light sensor . ​ 5. The method of claim 3, wherein the high dynamic range optical spectral imaging method is suitable for dynamic scenes. In S4, the maximum radiance value when the pixels of the highlight sensor and the low-light sensor are saturated is calculated, and the actual scene radiance range captured is calculated in combination with the minimum radiance of the highlight sensor and the low-light sensor. Computing maximum radiance values for highlight sensor pixels when saturated The formula is: ； wherein, is the maximum pixel value of the highlight sensor; is the number of electrons generated inside the highlight sensor; C is the conversion efficiency; A is the photosensitive area of the pixel of the front lens module; is the shading coefficient; F is the ratio of the equivalent focal length and the optical aperture of the combination of the front lens module and the variable ratio beam splitter module; h is the Planck constant; c is the speed of light; is the median wavelength of the incident light; is the transmittance of the combination of the front lens module and the variable ratio beam splitter module; is the quantum efficiency of the highlight sensor, is the exposure time of the highlight sensor; Maximum radiance value at which a low-light sensor pixel saturates The calculation formula is: ； wherein, is the maximum pixel value of the low light sensor; is the number of electrons excited within the low light sensor; is the exposure time of the low light sensor, is the quantum efficiency of the low light sensor; The actual received radiance range of the highlight sensor is determined as: ； Divide by the spectral ratio to obtain the actual scene radiance range captured by the highlight sensor: ； The actual received radiance range of the low-light sensor is determined as: ； Divide by the spectral ratio to obtain the actual scene radiance range captured by the low-light sensor: 。 6. The method of claim 5, wherein the high dynamic range optical spectral imaging method is suitable for dynamic scenes. In S5, the expression for judging whether the actual received radiance range of the highlight sensor and the low-light sensor completely covers the radiance range of the scene is: and ; The expression for judging whether the actual radiance range of the highlight sensor and the low-light sensor has an overlapping area is: 。 7. The high dynamic range optical beam splitting imaging method suitable for dynamic scenes according to claim 5, characterized in that, In S6, the exposure time of the low-light sensor is prolonged, and the actual received radiance range of the virtual imaging light path is compensated for the actual received radiance range of the low-light sensor. The specific process is as follows: The exposure time of the low-light sensor is extended for the first time i , The exposure time of the low-light sensor is extended for the first time , The exposure time of the low-light sensor is extended for the first time The exposure time of the low-light sensor is extended for the first time i The exposure time of the low-light sensor is extended for the first time ≥ 1.1; Range of radiance actually received by the virtual imaging light path Is: ； wherein, represents the minimum radiance of the virtual imaging light path obtained by changing the exposure time for the first i time, represents the maximum radiance of the virtual imaging light path obtained by changing the exposure time for the first i time. The expression for judging whether the actual received radiance range of the virtual imaging light path and the actual received radiance range of the low-light sensor have an overlap is as follows: whether ; If not If there is overlap, the range of luminance actually received by the low light sensor becomes: ； If yes No overlap, return to modify the exposure time of the extended low light sensor.

8. The method of claim 1, wherein the high dynamic range optical spectral imaging method is suitable for dynamic scenes. Adjusting the exposure time of the highlight sensor in S6 Specifically: reducing the exposure time of the highlight sensor by a factor.

9. The method of claim 1, wherein the method is suitable for dynamic scenes. In S1, the instrument for measuring the radiance range of the scene is a radiance meter.

Images