A method for correcting parasitic light response of a multi-exposure large dynamic range transient image

Abstract

The present application relates to a method for correcting parasitic light response of a CMOS image sensor, and in particular to a method for correcting parasitic light response of a multi-exposure large dynamic range transient image, which solves the problems of high complexity of existing parasitic light response correction systems and unsuitability of existing correction methods for transient imaging technology with fast changes in light intensity of an imaging target. The method can be implemented by using the original structure of a multi-exposure large dynamic range transient imaging system, which includes selecting a global shutter type CMOS image sensor and an image intensifier and coupling them together, establishing a residual light intensity expression formula with the readout time of each row of pixels of the global shutter type CMOS image sensor as the unit time according to the residual light intensity curve of the image intensifier, obtaining N frames of exposure images of the global shutter type CMOS image sensor within the residual light time of the image intensifier, and correcting the parasitic light response after removing the background of the images, so as to eliminate the parasitic light response and improve the accuracy of the obtained transient image.

Description

Technical Field

[0001] This invention relates to a parasitic light-sensing correction method for CMOS image sensors, specifically to a parasitic light-sensing correction method for multi-exposure large dynamic range transient images. Background Technology

[0002] In research fields such as bioimaging, biomedicine, and high-energy physics, the target images to be imaged often have a large dynamic range, which requires the imaging system to also have a large linear dynamic range. Currently, acquiring multiple images of different intensities through multiple exposures and fusing them to obtain images with a large linear dynamic range is a widely used imaging technique. However, ordinary multiple exposure techniques require the light intensity of the imaged target to remain constant or change very little, which is not achievable for transient imaging targets.

[0003] Achieving high temporal resolution using image intensifiers and employing multiple exposures within the afterglow decay time is an effective method for imaging large dynamic range transient processes. However, parasitic photosensitive elements caused by the photosensitive charge storage nodes of CMOS image sensors can cause the image to remain affected by illumination even after the exposure is complete. For transient imaging, short exposure times and long readout times exacerbate the impact of parasitic photosensitive elements on the image. Therefore, correcting parasitic photosensitive elements in multi-exposure, large dynamic range transient imaging is of great value for accurately acquiring transient images.

[0004] Currently, eliminating the effects of parasitic light sensing mainly involves special design of the image sensor, but this special design increases system complexity. In addition, there are a few correction methods based on acquired images, but these methods require the light intensity of the imaging object to remain constant, making them unsuitable for transient imaging where the light intensity of the imaging target changes rapidly. Summary of the Invention

[0005] The purpose of this invention is to solve the technical problems of high complexity of existing parasitic light sensing correction systems for multi-exposure large dynamic range transient images, or the inapplicability of existing correction methods to transient imaging where the light intensity of the imaging target changes rapidly, and to provide a parasitic light sensing correction method for multi-exposure large dynamic range transient images.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for correcting parasitic light sensitivity in multi-exposure, high dynamic range transient images, characterized by the following steps: Step 1: Select a global shutter type CMOS image sensor with m rows and n columns of pixels, and obtain the readout time t0 and parasitic light sensitivity of each row of pixels of the global shutter type CMOS image sensor, where m and n are both positive integers; Step 2: Select the image intensifier, obtain the afterglow time of the image intensifier and the afterglow intensity curve as a function of time, and calculate the maximum number of exposures N of the global shutter CMOS image sensor within the afterglow time of the image intensifier, where N is an integer and N > 2. Step 3: Based on the afterglow intensity curve obtained in Step 2, establish an expression for the afterglow intensity with the readout time t0 of each row of pixels of the global shutter CMOS image sensor as the unit time, and obtain the afterglow intensity Y(kt0) of the image intensifier in the kth time t0, where k is an integer not less than 0. Step 4: Couple the image intensifier to the global shutter type CMOS image sensor; Step 5: Expose the global shutter type CMOS image sensor N times during the afterglow time of the image intensifier to obtain N exposed images, and obtain the background image of the N exposed images; Step 6: Perform background removal processing on the N exposed images using their corresponding background images to obtain background-removed images I0, I1, ..., IN-1 of the N exposed images; Step 7: Based on the parasitic light sensitivity of the global shutter CMOS image sensor obtained in Step 1 and the afterglow intensity Y(kt0) of the image intensifier in the kth time t0 obtained in Step 3, parasitic light sensitivity correction is performed on the background-reduced images I0, I1, ..., IN-1 of the N frames of exposure images obtained in Step 6, respectively, to obtain the N frames of parasitic light sensitivity corrected images CI0, CI1, ..., CIN-1, thus completing the parasitic light sensitivity correction of multi-exposure large dynamic range transient images.

[0007] Further, in step 7, within the N frames of parasitic light-sensing corrected images CI0, CI1, ..., CIN-1, the value CIh of the first row of pixels in the (h+1)th frame of the parasitic light-sensing corrected image. 0,j Calculated using the following formula: CIh 0,j =Ih 0,j Where h is an integer, and 0 ≤ h ≤ N-1; j represents the column coordinate of the pixel, j is an integer, and 0≤j≤n-1; Ih 0,j This represents the value of the pixel at coordinates (0,j) in the background-removed image of the (h+1)th frame of the exposed image.

[0008] Further, in step 7, within the N frames of parasitic light-sensing corrected images CI0, CI1, ..., CIN-1, the pixel value CIh in the (i+1)th row of the (h+1)th frame of the parasitic light-sensing corrected image. i,j Calculated using the following formula: CIhi,j =Ih i,j (1-PLS i,j ×(Y((k 2h+1 +1)t0)+Y((k 2h+1 +2)t0)+…+Y((k 2h+1 +i)t0))) / (Y(k 2h t0)+Y((k 2h +1)t0)…+Y(k 2h+1 t0)+Y((k 2h+1 +1)t0)+Y((k 2h+1 +2)t0)+…+Y((k 2h+1 +i)t0)) Where i represents the row coordinate of the pixel, i is an integer, and 1≤i≤m-1; CIh i,j This represents the value of the pixel at coordinates (i,j) in the corrected image of frame h+1; Ih i,j This represents the value of the pixel at coordinates (i,j) in the background-removed image of the (h+1)th frame of the exposed image; PLS stands for parasitic light sensitivity of a global shutter CMOS image sensor. i,j This represents the parasitic light sensitivity of the pixel at coordinates (i,j) in a global shutter CMOS image sensor. k 2h k represents the number of unit times at the start of exposure of the (h+1)th frame of the image. 2h t0 represents the start time of exposure of the (h+1)th frame of the image; k 2h+1 k represents the number of unit times at the end of the exposure of the (h+1)th frame of the image. 2h+1 This indicates the end time of the exposure of the (h+1)th frame of the image; Y((k 2h+1 +i)t0) represents the image enhancer at the k-th... 2h+1 +i afterglow intensity within time interval t0.

[0009] Further, step 4 specifically involves coupling the image intensifier to the global shutter type CMOS image sensor via an optical cone or directly.

[0010] Furthermore, in step 1, the pixels of the global shutter type CMOS image sensor are provided with photosensitive areas and charge storage nodes.

[0011] Further, in step 1, the parasitic photosensitivity is the ratio of the photosensing efficiency of the charge storage node to the photosensing efficiency of the photosensitive region.

[0012] Furthermore, in step 5, the specific method for obtaining the background image of the N exposed images is as follows: before obtaining each exposed image, or after obtaining each exposed image, obtain its corresponding background image.

[0013] Further, in step 7, within the N frames of parasitic light-sensing corrected images CI0, CI1, ..., CIN-1, the value CI0 of the first row of pixels in the first frame of the parasitic light-sensing corrected image... 0,j Calculated using the following formula: CI0 0,j =I1 0,j Where j represents the column coordinate of the pixel, j is an integer, and 0≤j≤n-1; CI0 0,j This represents the value of the pixel at coordinates (0,j) in the first frame of the image after parasitic light sensing correction; I0 0,j This represents the value of the pixel at coordinates (0,j) in the background-removed image of the first exposed image; In the first frame of the image after parasitic light-sensing correction, the value CI0 of the second row of pixels is... 1,j Calculated using the following formula: CI0 1,j =I0 1,j (1-PLS 1,j ×Y((k1+1)t0)) / (Y(k0t0)+Y((k0+1)t0)+…+Y(k1t0)+Y((k1+1)t0)); Among them, CI0 1,j This represents the value of the pixel at coordinates (1,j) in the first frame of the image after parasitic light sensing correction; I0 1,j This represents the value of the pixel at coordinates (1,j) in the background-removed image of the first exposed image; PLS 1,j This represents the parasitic light sensitivity of the pixel at coordinates (1,j) in a global shutter CMOS image sensor. k0 represents the number of units of time at the start of the exposure of the first frame of the image, and k0t0 represents the start time of the exposure of the first frame of the image. k1 represents the number of units of time at the end of the exposure of the first frame of the image, and k1t0 represents the end of the exposure of the first frame of the image. Y((k1+1)t0) represents the afterglow intensity of the image intensifier during the k1+1 t0 time period; In the first frame of the image after parasitic light sensing correction, the value of the third row of pixels is CI0. 2,j Calculated using the following formula: CI0 2,j =I0 2,j (1-PLS 2,j ×(Y((k1+1)t0)+Y((k1+2)t0))) / (Y(k0t0)+Y((k0+1)t0)+ …+Y(k1t0)+Y((k1+1)t0)+Y((k1+2)t0)); Among them, CI0 2,j This represents the value of the pixel at coordinates (2,j) in the first frame of the image after parasitic light sensing correction; I0 2,j This represents the value of the pixel at coordinates (2,j) in the background-removed image of the first exposed image; PLS 2,j This represents the parasitic light sensitivity of the pixel at coordinates (2,j) in a global shutter CMOS image sensor. In the first frame of the image after parasitic light-sensing correction, the value CI0 of the pixel in the nth row is... n-1,j Calculated using the following formula: CI0 n-1,j =I0 n-1,j (1-PLS n-1,j ×(Y((k1+1)t0)+Y((k1+2)t0)+…+Y((k1+n-1)t0))) / (Y(k0t0)+Y((k0+1)t0)…+Y(k1t0)+Y((k1+1)t0)+Y((k1+2)t0)+…+Y((k1+n-1)t0)); Among them, CI0 n-1,j This represents the value of the pixel at coordinates (n-1,j) in the first frame of the image after parasitic light sensing correction; I0 n-1,j This represents the value of the pixel at coordinates (n-1,j) in the background-removed image of the first exposed image; PLS n-1,j This represents the parasitic light sensitivity of the pixel at coordinates (n-1,j) in a global shutter CMOS image sensor.

[0014] Further, in step 7, within the N frames of parasitic light-sensing corrected images CI0, CI1, ..., CIN-1, the value CI1 of the first row of pixels in the second frame of the parasitic light-sensing corrected image... 0,j Calculated using the following formula: CI1 0,j =I1 0,j Where j represents the column coordinate of the pixel, j is an integer, and 0≤j≤n-1; CI1 0,jThis represents the value of the pixel at coordinates (0,j) in the second frame of the image after parasitic light sensing correction; I1 0,j This represents the value of the pixel at coordinates (0,j) in the background-removed image of the second exposed image; In the second frame of the image after parasitic light sensing correction, the value CI1 of the second row of pixels is... 1,j Calculated using the following formula: CI1 1,j =I1 1,j (1-PLS 1,j ×Y((k3+1)t0)) / (Y(k2t0)+Y((k2+1)t0)+…+Y(k3t0)+Y((k3+1)t0)); Among them, CI1 1,j This represents the value of the pixel at coordinates (1,j) in the second frame of the image after parasitic light sensing correction; I1 1,j This represents the value of the pixel at coordinates (1,j) in the background-removed image of the second exposed image; PLS 1,j This represents the parasitic light sensitivity of the pixel at coordinates (1,j) in a global shutter CMOS image sensor. k2 represents the number of units of time at the start of the exposure of the second frame image, and k2t0 represents the start time of the exposure of the second frame image. k3 represents the number of unit time units at the end of the exposure of the second frame image, and k2t0 represents the end of the exposure of the second frame image. Y((k3+1)t0) represents the afterglow intensity of the image intensifier during the k3+1 t0 time period; In the second frame of the image after parasitic light sensing correction, the value CI1 of the third row of pixels is... 2,j Calculated using the following formula: CI1 2,j =I1 2,j (1-PLS 2,j ×(Y((k3+1)t0)+Y((k3+2)t0))) / (Y(k2t0)+Y((k2+1)t0)+…+Y(k3t0)+Y((k3+1)t0)+Y((k3+2)t0)); Among them, CI1 2,j This represents the value of the pixel at coordinates (2,j) in the second frame of the image after parasitic light sensing correction; I1 2,j This represents the value of the pixel at coordinates (2,j) in the background-removed image of the second exposed image; PLS 2,jThis represents the parasitic light sensitivity of the pixel at coordinates (2,j) in a global shutter CMOS image sensor. In the second frame of the image after parasitic light-sensing correction, the value CI1 of the pixel in the nth row is... 2,j Calculated using the following formula: CI1 n-1,j =I1 n-1,j (1-PLS n-1,j ×(Y((k3+1)t0)+Y((k3+2)t0)+…+Y((k3+n-1)t0))) / (Y(k2t0)+Y((k2+1)t0)+…+Y(k3t0)+Y((k3+1)t0)+Y((k3+2)t0)+…+Y((k3+n-1)t0)); Among them, CI1 n-1,j This represents the value of the pixel at coordinates (n-1,j) in the second frame of the image after parasitic light sensing correction; I1 n-1,j This represents the value of the pixel at coordinates (n-1,j) in the background-removed image of the second exposed image; PLS n-1,j This represents the parasitic light sensitivity of the pixel at coordinates (n-1,j) in a global shutter CMOS image sensor.

[0015] Compared with the prior art, the present invention has the following beneficial technical effects: This invention provides a method for correcting parasitic light sensitivity in multi-exposure large dynamic range transient images. The correction can be achieved through the existing structure of the multi-exposure large dynamic range transient imaging system without increasing the system complexity. By utilizing the characteristic that the afterglow intensity curve of the image intensifier remains stable in each exposure, the parasitic light sensitivity effect of the subsequent afterglow of the image intensifier on the current image is quantified, thereby eliminating parasitic light sensitivity through image correction and improving the accuracy of the acquired transient images. Attached Figure Description

[0016] Figure 1 A flowchart of a parasitic light-sensing correction method for multi-exposure large dynamic range transient images provided by the present invention; Figure 2 This is a graph showing the change of afterglow intensity over time corresponding to the afterglow intensity expression obtained in step 3 of this embodiment of the invention. Detailed Implementation

[0017] The parasitic light sensitivity correction method for multi-exposure large dynamic range transient images proposed in this invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art should understand that these embodiments are merely used to explain the technical principles of this invention and are not intended to limit the scope of protection of this invention.

[0018] A method for parasitic light sensitivity correction of multi-exposure large dynamic range transient images, such as Figure 1 As shown, it includes the following steps: Step 1: Select a global shutter type CMOS image sensor with m rows and n columns of pixels, and obtain the readout time t0 and parasitic light sensitivity of each row of pixels in the global shutter type CMOS image sensor, where m and n are both positive integers. The pixels of the global shutter type CMOS image sensor are configured with photosensitive areas and charge storage nodes. The parasitic light sensitivity is the ratio of the photosensitive efficiency of the charge storage nodes to that of the photosensitive areas.

[0019] Step 2: Select the image intensifier, obtain the afterglow time of the image intensifier and the afterglow intensity curve as a function of time, and calculate the maximum number of exposures N of the global shutter CMOS image sensor within the afterglow time of the image intensifier, where N is an integer and N > 2.

[0020] Step 3: Based on the afterglow intensity curve, establish an expression for the afterglow intensity with the readout time t0 of each row of pixels in the global shutter CMOS image sensor as the unit time. This yields the afterglow intensity Y(kt0) of the image intensifier during the kth time interval t0, where k is an integer not less than 0. The afterglow intensity curve is shown in the figure below. Figure 2 As shown.

[0021] Step 4: Couple the image intensifier to the global shutter type CMOS image sensor via an optical cone or directly.

[0022] Step 5: Expose the global shutter CMOS image sensor N times within the afterglow time of the image intensifier to obtain N exposed images, and obtain the background images of the N exposed images. Before performing background removal processing on the N exposed images, the background images of the N exposed images need to be obtained either before or after the effective exposure. That is, the background image of each exposed image is obtained either before or after obtaining its corresponding exposed image.

[0023] Step 6: Perform background removal processing on the N exposed images using their corresponding background images to obtain background-removed images I0, I1, ..., IN-1 of the N exposed images.

[0024] Step 7: Based on the parasitic light sensitivity of the global shutter CMOS image sensor obtained in Step 1 and the afterglow intensity Y(kt0) of the image intensifier in the kth time t0 obtained in Step 3, parasitic light sensitivity correction is performed on the background-reduced images I0, I1, ..., IN-1 of the N frames of exposure images obtained in Step 6, respectively, to obtain the N frames of parasitic light sensitivity corrected images CI0, CI1, ..., CIN-1, thus completing the parasitic light sensitivity correction of multi-exposure large dynamic range transient images.

[0025] For the (h+1)th frame of the image after parasitic light sensing correction, its first row pixel value CIh 0,j for: CIh 0,j =Ih 0,j Where h is an integer, and 0 ≤ h ≤ N-1; j represents the column coordinate of the pixel, j is an integer, and 0≤j≤n-1; Ih 0,j This represents the value of the pixel at coordinates (0,j) in the background-removed image of the (h+1)th frame of the exposed image; The pixel value CIh in the (i+1)th row i,j for: CIh i,j =Ih i,j (1-PLS i,j ×(Y((k 2h+1 +1)t0)+Y((k 2h+1 +2)t0)+…+Y((k 2h+1 +i)t0))) / (Y(k 2h t0)+Y((k 2h +1)t0)…+Y(k 2h+1 t0)+Y((k 2h+1 +1)t0)+Y((k 2h+1 +2)t0)+…+Y((k 2h+1 +i)t0)) Where i represents the row coordinate of the pixel, i is an integer, and 1≤i≤m-1; k 2h k represents the number of unit times at the start of exposure of the (h+1)th frame of the image. 2h+1 This represents the number of units of time at the end of the exposure of the (h+1)th frame of the image; CIh i,j This represents the value of the pixel at coordinates (i,j) in the corrected image of frame h+1; PLS stands for parasitic light sensitivity of a global shutter CMOS image sensor. i,jThis represents the parasitic light sensitivity of the pixel at coordinates (i,j) in a global shutter CMOS image sensor.

[0026] The exposure start times of the N frames of exposed images are denoted as k0t0, k2t0, ..., k. 2N-2 t0, and the end times of the exposure are denoted as k1t0, k3t0, ..., k 2N-1 t0, where k0, k1, k2, k3…k 2N-2 and k 2N-1 All are integers not less than 0, k0t0, k1t0, k2t0, k3t0…k 2N-2 t0, k 2N-1 t0 represents the k0th, k1th, k2th, k3th, ..., kth digits respectively. 2N-2 k 2N-1 Each t0 time.

[0027] Specifically, for the first frame of the image after parasitic light sensing correction CI0, that is, when h=0, the value of the first row of pixels CI0 0,j for: CI0 0,j =I1 0,j Where j represents the column coordinate of a pixel, j is an integer, and 0 ≤ j ≤ n-1; CI0 0,j I0 represents the value of the pixel at coordinates (0,j) in the first frame of the image after parasitic light sensing correction. 0,j This represents the value of the pixel at coordinates (0,j) in the background-removed image of the first exposed image.

[0028] The value of the second row of pixels is CI0. 1,j for: CI0 1,j =I0 1,j (1-PLS 1,j ×Y((k1+1)t0)) / ( Y(k0t0)+ Y((k0+1)t0) …+ Y(k1t0)+ Y((k1+1)t0)); Among them, CI0 1,j I0 represents the value of the pixel at coordinates (1,j) in the first frame of the image after parasitic light sensing correction. 1,j PLS represents the value of the pixel at coordinates (1,j) in the background-removed image of the first exposed image. 1,j This represents the parasitic light sensitivity of the pixel at coordinates (1,j) in a global shutter CMOS image sensor. k0 represents the number of units of time at the start of the exposure of the first frame of the image, k0t0 represents the start time of the exposure of the first frame of the image; k1 represents the number of units of time at the end of the exposure of the first frame of the image, k1t0 represents the end time of the exposure of the first frame of the image. Y((k1+1)t0) represents the afterglow intensity of the image intensifier during the k1+1 t0 time period.

[0029] The value of the third row of pixels is CI0. 2,j for: CI0 2,j =I0 2,j (1- PLS 2,j ×(Y((k1+1)t0)+Y((k1+2)t0))) / (Y(k0t0)+ Y((k0+1)t0) …+ Y(k1t0)+Y((k1+1)t0)+Y((k1+2)t0)); Among them, CI0 2,j I0 represents the value of the pixel at coordinates (2,j) in the first frame of the image after parasitic light sensing correction. 2,j PLS represents the value of the pixel at coordinates (2,j) in the background-removed image of the first exposed image. 2,j This represents the parasitic light sensitivity of the pixel at coordinates (2,j) in a global shutter CMOS image sensor.

[0030] And so on, the value of the pixel in the nth row is CI0. n-1,j for: CI0 n-1,j =I0 n-1,j (1-PLS n-1,j ×(Y((k1+1)t0)+Y((k1+2)t0)+…+Y((k1+n-1)t0))) / (Y(k0t0)+Y((k0+1)t0)…+Y(k1t0)+Y((k1+1)t0)+Y((k1+2)t0)+…+Y((k1+n-1)t0)); Among them, CI0 n-1,j I0 represents the value of the pixel at coordinates (n-1,j) in the first frame of the image after parasitic light sensing correction. n-1,j PLS represents the value of the pixel at coordinates (n-1,j) in the background-removed image of the first exposed image. n-1,j This represents the parasitic light sensitivity of the pixel at coordinates (n-1,j) in a global shutter CMOS image sensor.

[0031] For the parasitic light-sensing corrected image CI1 in the second frame, i.e., when h=1, the value of the first row of pixels is CI1. 0,j for: CI10,j =I1 0,j Where j represents the column coordinate of a pixel, j is an integer, and 0 ≤ j ≤ n-1; CI1 0,j I1 represents the value of the pixel at coordinates (0,j) in the second frame of the image after parasitic light sensing correction. 0,j This represents the value of the pixel at coordinates (0,j) in the background-removed image of the second exposed image.

[0032] The value of the second row of pixels is CI1. 1,j for: CI1 1,j =I1 1,j (1- PLS 1,j ×Y((k3+1)t0)) / ( Y(k2t0)+ Y((k2+1)t0) +…+ Y(k3t0)+ Y((k3+1)t0)); Among them, CI1 1,j I1 represents the value of the pixel at coordinates (1,j) in the second frame of the image after parasitic light sensing correction. 1,j PLS represents the value of the pixel at coordinates (1,j) in the background-removed image of the second exposed image. 1,j This represents the parasitic light sensitivity of the pixel at coordinates (1,j) in a global shutter CMOS image sensor. k2 represents the number of units of time at the start of the exposure of the second frame image, and k2t0 represents the start time of the exposure of the second frame image; k3 represents the number of units of time at the end of the exposure of the second frame image, and k2t0 represents the end time of the exposure of the second frame image. Y((k3+1)t0) represents the afterglow intensity of the image intensifier during the k3+1 t0 time period.

[0033] The value of the third row of pixels is CI1. 2,j for: CI1 2,j =I1 2,j (1- PLS 2,j ×(Y((k3+1)t0)+Y((k3+2)t0))) / (Y(k2t0)+ Y((k2+1)t0)+…+ Y(k3t0)+Y((k3+1)t0)+Y((k3+2)t0)); Among them, CI1 2,j I1 represents the value of the pixel at coordinates (2,j) in the second frame of the image after parasitic light sensing correction. 2,j PLS represents the value of the pixel at coordinates (2,j) in the background-removed image of the second exposed image. 2,jThis represents the parasitic light sensitivity of the pixel at coordinates (2,j) in a global shutter CMOS image sensor.

[0034] And so on, the value CI1 of the pixel in the nth row 2,j for: CI1 n-1,j =I1 n-1,j (1- PLS n-1,j ×(Y((k3+1)t0)+Y((k3+2)t0)+…+Y((k3+n-1)t0))) / (Y(k2t0)+Y((k2+1)t0)+…+Y(k3t0)+Y((k3+1)t0)+Y((k3+2)t0)+…+Y((k3+n-1)t0)); Among them, CI1 n-1,j I1 represents the value of the pixel at coordinates (n-1,j) in the second frame of the image after parasitic light sensing correction. n-1,j PLS represents the value of the pixel at coordinates (n-1,j) in the background-removed image of the second exposed image. n-1,j This represents the parasitic light sensitivity of the pixel at coordinates (n-1,j) in a global shutter CMOS image sensor.

Claims

1. A method for correcting parasitic light sensitivity in multi-exposure, large dynamic range transient images, characterized in that, Includes the following steps: Step 1: Select a global shutter type CMOS image sensor with m rows and n columns of pixels, and obtain the readout time t0 and parasitic light sensitivity of each row of pixels of the global shutter type CMOS image sensor, where m and n are both integers greater than or equal to 1; Step 2: Select the image intensifier, obtain the afterglow time of the image intensifier and the afterglow intensity curve as a function of time, and calculate the maximum number of exposures N of the global shutter CMOS image sensor within the afterglow time of the image intensifier, where N is an integer and N > 2. Step 3: Based on the afterglow intensity curve obtained in Step 2, establish an expression for the afterglow intensity with the readout time t0 of each row of pixels of the global shutter CMOS image sensor as the unit time, and obtain the afterglow intensity Y(kt0) of the image intensifier in the kth time t0, where k is an integer not less than 0. Step 4: Couple the image intensifier to the global shutter type CMOS image sensor; Step 5: Expose the global shutter type CMOS image sensor N times during the afterglow time of the image intensifier to obtain N exposed images, and obtain the background image of the N exposed images; Step 6: Perform background removal processing on the N exposed images using their corresponding background images to obtain background-removed images I0, I1, ..., IN-1 of the N exposed images; Step 7: Based on the parasitic light sensitivity of the global shutter CMOS image sensor obtained in Step 1 and the afterglow intensity Y(kt0) of the image intensifier in the kth time t0 obtained in Step 3, parasitic light sensitivity correction is performed on the background-reduced images I0, I1, ..., IN-1 of the N frames of exposure images obtained in Step 6, respectively, to obtain the N frames of parasitic light sensitivity corrected images CI0, CI1, ..., CIN-1, thus completing the parasitic light sensitivity correction of multi-exposure large dynamic range transient images.

2. The method for parasitic light sensitivity correction of multi-exposure large dynamic range transient images according to claim 1, characterized in that: In step 7, within the N frames of parasitic light-sensing corrected images CI0, CI1, ..., CIN-1, the value of the first row of pixels CIh in the (h+1)th frame of the parasitic light-sensing corrected image CIh is... 0,j Calculated using the following formula: CIh 0,j =Ih 0,j Where h is an integer, and 0 ≤ h ≤ N-1; j represents the column coordinate of the pixel, j is an integer, and 0≤j≤n-1; Ih 0,j This represents the value of the pixel at coordinates (0,j) in the background-removed image of the (h+1)th frame of the exposed image.

3. The method for parasitic light sensitivity correction of multi-exposure large dynamic range transient images according to claim 2, characterized in that: In step 7, within the N frames of parasitic light-sensing corrected images CI0, CI1, ..., CIN-1, the pixel value CIh in the (h+1)th frame of the parasitic light-sensing corrected image CIh is the pixel value CIh in the (i+1)th row. i,j Calculated using the following formula: CIh i,j =Ih i,j (1-PLS i,j ×(Y((k 2h+1 +1)t0)+Y((k 2h+1 +2)t0)+…+Y((k 2h+1 +i)t0))) / (Y(k 2h t0)+Y((k 2h +1)t0)…+Y(k 2h+1 t0)+Y((k 2h+1 +1)t0)+Y((k 2h+1 +2)t0)+…+Y((k 2h+1 +i)t0)) Where i represents the row coordinate of the pixel, i is an integer, and 1≤i≤m-1; CIh i,j This represents the value of the pixel at coordinates (i,j) in the corrected image of frame h+1; Ih i,j This represents the value of the pixel at coordinates (i,j) in the background-removed image of the (h+1)th frame of the exposed image; PLS stands for parasitic light sensitivity of a global shutter CMOS image sensor. i,j This represents the parasitic light sensitivity of the pixel at coordinates (i,j) in a global shutter CMOS image sensor. k 2h k represents the number of unit times at the start of exposure of the (h+1)th frame of the image. 2h t0 represents the start time of exposure of the (h+1)th frame of the image; k 2h+1 k represents the number of unit times at the end of the exposure of the (h+1)th frame of the image. 2h+1 This indicates the end time of the exposure of the (h+1)th frame of the image; Y((k 2h+1 +i)t0) represents the image enhancer at the k-th... 2h+1 +i afterglow intensity within time interval t0.

4. The method for parasitic light sensitivity correction of multi-exposure large dynamic range transient images according to claim 3, characterized in that, Step 4 specifically involves coupling the image intensifier to the global shutter type CMOS image sensor via an optical cone or directly.

5. The method for parasitic light sensitivity correction of multi-exposure large dynamic range transient images according to claim 4, characterized in that: In step 1, the pixels of the global shutter type CMOS image sensor are provided with photosensitive areas and charge storage nodes.

6. The method for parasitic light sensitivity correction of multi-exposure large dynamic range transient images according to claim 5, characterized in that: In step 1, the parasitic photosensitivity is the ratio of the photosensing efficiency of the charge storage node to the photosensing efficiency of the photosensitive region.

7. The method for parasitic light sensitivity correction of multi-exposure large dynamic range transient images according to claim 6, characterized in that, In step 5, the specific method for obtaining the background image of N exposed images is as follows: before obtaining each exposed image, or after obtaining each exposed image, obtain its corresponding background image.

8. A method for parasitic light sensitivity correction of multi-exposure large dynamic range transient images according to any one of claims 3-7, characterized in that: In step 7, within the N frames of parasitic light-sensing corrected images CI0, CI1, ..., CIN-1, the value of the first row of pixels in the first frame of the parasitic light-sensing corrected image CI0 is CI0. 0,j Calculated using the following formula: CI0 0,j =I0 0,j Where j represents the column coordinate of the pixel, j is an integer, and 0≤j≤n-1; CI0 0,j This represents the value of the pixel at coordinates (0,j) in the first frame of the image after parasitic light sensing correction; I0 0,j This represents the value of the pixel at coordinates (0,j) in the background-removed image of the first exposed image; In the first frame of the parasitic light-sensing corrected image CI0, the value of the second row of pixels is CI0. 1,j for: CIO 1,j =I0 1,j (1-PLS 1,j ×Y((k1+1)t0)) / (Y(k0t0)+Y((k0+1)t0) …+Y(k1t0)+Y((k1+1)t0)); Among them, CI0 1,j This represents the value of the pixel at coordinates (1,j) in the first frame of the image after parasitic light sensing correction; I0 1,j This represents the value of the pixel at coordinates (1,j) in the background-removed image of the first exposed image; PLS 1,j This represents the parasitic light sensitivity of the pixel at coordinates (1,j) in a global shutter CMOS image sensor. k0 represents the number of units of time at the start of the exposure of the first frame of the image, and k0t0 represents the start time of the exposure of the first frame of the image. k1 represents the number of units of time at the end of the exposure of the first frame of the image, and k1t0 represents the end of the exposure of the first frame of the image. Y((k1+1)t0) represents the afterglow intensity of the image intensifier during the k1+1 t0 time period; In the first frame of the parasitic light-sensing corrected image CI0, the value of the third row of pixels is CI0. 2,j Calculated using the following formula: CIO 2,j =I0 2,j (1-PLS 2,j ×(Y((k1+1)t0)+Y((k1+2)t0))) / (Y(k0t0)+Y((k0+1)t0)…+ Y(k1t0)+Y((k1+1)t0)+Y((k1+2)t0)); Among them, CI0 2,j This represents the value of the pixel at coordinates (2,j) in the first frame of the image after parasitic light sensing correction; I0 2,j This represents the value of the pixel at coordinates (2,j) in the background-removed image of the first exposed image; PLS 2,j This represents the parasitic light sensitivity of the pixel at coordinates (2,j) in a global shutter CMOS image sensor. In the first frame of the parasitic light-sensing corrected image CI0, the value of the pixel in the nth row is CI0. n-1,j Calculated using the following formula: CIO n-1,j =I0 n-1,j (1-PLS n-1,j ×(Y((k1+1)t0)+Y((k1+2)t0)+…+Y((k1+n-1)t0))) / (Y(k0t0)+Y((k0+1)t0)…+Y(k1t0)+Y((k1+1)t0)+Y((k1+2)t0)+…+Y((k1+n-1)t0)); Among them, CI0 n-1,j This represents the value of the pixel at coordinates (n-1,j) in the first frame of the image after parasitic light sensing correction; I0 n-1,j This represents the value of the pixel at coordinates (n-1,j) in the background-removed image of the first exposed image; PLS n-1,j This represents the parasitic light sensitivity of the pixel at coordinates (n-1,j) in a global shutter CMOS image sensor.

9. A method for parasitic light sensitivity correction of multi-exposure large dynamic range transient images according to any one of claims 3-7, characterized in that: In step 7, within the N frames of parasitic light-sensing corrected images CI0, CI1, ..., CIN-1, in the second frame of the parasitic light-sensing corrected image CI1, the value of the first row of pixels is CI1. 0,j Calculated using the following formula: CI1 0,j =I1 0,j Where j represents the column coordinate of the pixel, j is an integer, and 0≤j≤n-1; CI1 0,j This represents the value of the pixel at coordinates (0,j) in the second frame of the image after parasitic light sensing correction; I1 0,j This represents the value of the pixel at coordinates (0,j) in the background-removed image of the second exposed image; In the second frame of the parasitic light-sensing corrected image CI1, the value of the second row of pixels is CI1. 1,j Calculated using the following formula: CI1 1,j =I1 1,j (1-PLS 1,j ×Y((k3+1)t0)) / (Y(k2t0)+Y((k2+1)t0)+…+ Y(k3t0)+ Y((k3+1)t0)); Among them, CI1 1,j This represents the value of the pixel at coordinates (1,j) in the second frame of the image after parasitic light sensing correction; I1 1,j This represents the value of the pixel at coordinates (1,j) in the background-removed image of the second exposed image; PLS 1,j This represents the parasitic light sensitivity of the pixel at coordinates (1,j) in a global shutter CMOS image sensor. k2 represents the number of units of time at the start of the exposure of the second frame image, and k2t0 represents the start time of the exposure of the second frame image. k3 represents the number of unit time units at the end of the exposure of the second frame image, and k2t0 represents the end of the exposure of the second frame image. Y((k3+1)t0) represents the afterglow intensity of the image intensifier during the k3+1 t0 time period; In the second frame of the parasitic light-sensing corrected image CI1, the value of the third row of pixels is CI1. 2,j Calculated using the following formula: CI1 2,j =I1 2,j (1-PLS 2,j ×(Y((k3+1)t0)+Y((k3+2)t0))) / (Y(k2t0)+Y((k2+1)t0)+…+ Y(k3t0)+Y((k3+1)t0)+Y((k3+2)t0)); Among them, CI1 2,j This represents the value of the pixel at coordinates (2,j) in the second frame of the image after parasitic light sensing correction; I1 2,j This represents the value of the pixel at coordinates (2,j) in the background-removed image of the second exposed image; PLS 2,j This represents the parasitic light sensitivity of the pixel at coordinates (2,j) in a global shutter CMOS image sensor. In the second frame of the parasitic light-sensing corrected image CI1, the value of the pixel in the nth row is CI1. 2,j Calculated using the following formula: CI1 n-1,j =I1 n-1,j (1-PLS n-1,j ×(Y((k3+1)t0)+Y((k3+2)t0)+…+Y((k3+n-1)t0))) / (Y(k2t0)+Y((k2+1)t0)+…+Y(k3t0)+Y((k3+1)t0)+Y((k3+2)t0)+…+Y((k3+n-1)t0)); Among them, CI1 n-1,j This represents the value of the pixel at coordinates (n-1,j) in the second frame of the image after parasitic light sensing correction; I1 n-1,j This represents the value of the pixel at coordinates (n-1,j) in the background-removed image of the second exposed image; PLS n-1,j This represents the parasitic light sensitivity of the pixel at coordinates (n-1,j) in a global shutter CMOS image sensor.

Images