A method for realizing single-frame large dynamic imaging based on composite dielectric grating photosensitive detector

Abstract

The present invention discloses a method for realizing single-frame large dynamic imaging based on a composite dielectric grating photosensitive detector, and belongs to the field of semiconductor technology. The method comprises: using two different word line selection modules to control odd rows and even rows respectively, so that reset, exposure, reading and waiting voltages with different time lengths are applied to odd rows or even rows, and the exposure time of odd rows and even rows is set to different lengths. After the composite dielectric grating photosensitive detector pixel array is read out, the pixels of the odd rows and even rows are interpolated and image fused to realize single-frame large dynamic imaging. The present invention can realize single-frame large dynamic imaging of the composite dielectric grating photosensitive detector pixel array, improve the temporal resolution of large dynamic imaging, reduce the motion blur of large dynamic fusion of moving images, and reduce the cost of the registration algorithm required for multi-frame fusion. This technology can be applied to scenes with high temporal resolution, especially for fast-moving objects.

Description

Technical Field

[0001] The invention relates to a method for realizing single-frame large dynamic imaging based on a composite dielectric grating photosensitive detector, and belongs to the technical field of semiconductors. Background Art

[0002] High Dynamic Range Imaging (HDR Imaging) is an image processing technology designed to capture and display a wider range of brightness than traditional imaging technology. This technology can process scenes from very dark to very bright, so that the final image can more realistically reflect the brightness differences seen by the human eye in the real world; and can retain more details.

[0003] At present, the main means to achieve large dynamic imaging are multi-frame fusion and single exposure HDR. Among them, the method of achieving large dynamic imaging with multi-frame fusion requires a higher frame rate and readout speed. For scenes with higher temporal resolution, especially for fast-moving objects, large dynamic imaging with multi-frame fusion is difficult to avoid the problems of blur and ghosting that cannot be eliminated in alignment and mapping. Traditional photosensitive detectors such as CCD and CMOS use multi-frame fusion to achieve large dynamic imaging. The inconsistent difference in time scale between the long exposure frame and the short exposure frame of the image causes image quality damage and sacrifices advantages such as frame rate. Single exposure HDR captures information of different brightness areas in a single exposure, avoids the loss of image spatial resolution through image interpolation and fusion, and then generates HDR images through algorithm processing. This method can solve the above-mentioned problems when multi-frame fusion is used to achieve large dynamic imaging.

[0004] Patent CN102938409A proposes a composite dielectric grating photosensitive detector, which is characterized in that a single semiconductor device can realize the complete reset, photosensitive and readout functions, constitute a complete pixel, and can greatly improve the pixel fill factor. As a new generation of imaging devices, this composite dielectric grating photosensitive detector has a faster working speed, a larger fill factor, more full well charges, and can be integrated with CMOS technology. Combined with the timing of the rolling shutter, the reset, photosensitive and readout function modes can be configured through the gating module. However, composite dielectric grating imaging arrays generally have the characteristics of small pixels and large arrays, and the amount of single-frame image data is large. If the traditional multi-frame image large dynamic range fusion is used, a higher registration algorithm computing power and high power consumption are required; in addition, in some application scenarios such as large dynamic imaging of fast-moving targets, multi-frame image large dynamic range fusion still has difficult-to-eliminate ghost problems. Single-frame large dynamic imaging can solve these problems, but there is little research on single-frame large dynamic imaging of photosensitive detectors such as CCD and CMOS, and there is still no research on single-frame large dynamic imaging of composite dielectric grating photosensitive detectors. Summary of the invention

[0005] In order to solve the existing problems, the present invention provides a method for realizing single-frame large dynamic imaging based on a composite dielectric grating photosensitive detector. The method is applied to a pixel array composed of composite dielectric grating photosensitive detectors, and the top control gates of the detectors in the same row are connected to form a word line WL, the drain ends of the detectors in the same column are connected to form a bit line BL, and the source ends are connected to form a source line SL; the method comprises:

[0006] Step 1, based on the odd-numbered row and even-numbered row word line selection modules, the odd-numbered row and even-numbered row of the detector are controlled to perform exposure of different time lengths respectively, so as to obtain long exposure image data and short exposure image data;

[0007] Step 2, interpolating the long exposure picture data and the short exposure picture data along the vertical word line direction to obtain the interpolated long exposure picture data and short exposure picture data;

[0008] Step 3: Fusing the interpolated long-exposure image data and short-exposure image data to obtain a single-frame large dynamic image.

[0009] Optionally, in step 2, a Lanczos kernel function is used to calculate a corresponding weight for each pixel in the long-exposure image data and the short-exposure image data, and the weights of all pixels are multiplied by the interpolation values ​​to obtain a final value of the interpolation point.

[0010] Optionally, step 2 includes:

[0011] Step 2.1, read the odd and even rows of image data, recorded as and ,in and are the total number of rows and columns of detectors, respectively;

[0012] Step 2.2: The odd-numbered and even-numbered image data are converted into Lanczos interpolation of the pixel neighborhood to obtain the interpolated image odd-numbered row matrix and the matrix with even rows ;

[0013] Step 2.3, change the odd-numbered rows of the matrix and the matrix with even rows Do it separately Transformation, the top-level images of the Laplace pyramid are recorded as and ;

[0014] Step 2.4, take a size of The neighborhood is centered on each pixel and has a neighborhood size of Calculate separately and The gradient mean corresponding to each pixel in is taken as the gray value of the pixel;

[0015] Step 2.5, get the fused image based on the gray value of each pixel, and compare and The larger value is used as the value of the corresponding pixel.

[0016] Optionally, the Lanczos interpolation kernel function is:

[0017]

[0018] in, Represents the smoothing coefficient.

[0019] Optionally, the image odd-numbered row matrix after interpolation in step 2.2 and the matrix with even rows for:

[0020]

[0021]

[0022] in, Indicates the odd-numbered lines of image data read. Indicates the even-numbered lines of image data read.

[0023] Optionally, in the pixel array, odd-numbered row word lines correspond one-to-one to output ends of odd-numbered row word line gating modules, and even-numbered row word lines correspond one-to-one to output ends of even-numbered row word line gating modules;

[0024] Set different row word lines to apply a negative bias voltage, a zero bias voltage and a ramp voltage in sequence under the control of the word line gating module to drive the corresponding row detector to enter the reset mode, exposure mode and readout mode respectively. The timing cycle T of the word line gating modules of different rows in the functional gating mode is the same, and the starting time of the timing cycle T of different row word lines has a fixed interval .

[0025] Optionally, the timing period T includes reset time, exposure time, readout time and waiting time for voltage application; the method controls the waiting time for voltage application or reset time to be different through the odd-row word line and the odd-row word line selection module so as to make the exposure time different.

[0026] Optionally, the composite dielectric grating photosensitive detector includes a MOS-C part and a MOSFET part, and the two parts are formed on the same P-type semiconductor substrate; wherein the MOS-C part includes a first dielectric layer, a charge coupling layer, a second dielectric layer and a first control gate stacked in sequence above the P-type semiconductor substrate; the MOSFET part includes a first dielectric layer, a charge coupling layer, a second top dielectric layer and a second control gate stacked in sequence above the P-type semiconductor substrate, an N-type source region and an N-type drain region are provided in the P-type semiconductor substrate and on one side close to the bottom dielectric layer, and a threshold adjustment injection region is provided in the P-type semiconductor substrate and below the bottom dielectric layer.

[0027] The beneficial effects of the present invention are:

[0028] The present invention is directed to a pixel array composed of composite dielectric grating photosensitive detectors. By connecting the top control gates of the detectors in the same row to form a word line WL, connecting the drain ends of the detectors in the same column to form a bit line BL, and connecting the source ends to form a source line SL, an odd-row word line gating module and an even-row word line gating module are used to apply reset, exposure, reading and waiting voltages of different time lengths to the odd-row and even-row detectors respectively, and the exposure time of the odd-row and even-row is set to different lengths, thereby obtaining long-exposure image data and short-exposure image data, and then using an interpolation method to fuse the long-exposure image data and the short-exposure image data, and realizing single-frame large-dynamic imaging; the scheme can realize single-frame large-dynamic imaging of the composite dielectric grating photosensitive detector pixel array, improve the time resolution of large-dynamic imaging, reduce the motion blur of large-dynamic fusion of motion images, and reduce the cost of the registration algorithm required for multi-frame fusion. This technology can be applied to scenes with high time resolution, especially for fast-moving objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 It is a schematic diagram of an odd-even word line gating module in a composite dielectric grating photosensitive detector array.

[0031] Figure 2 It is a word line timing diagram of a composite dielectric grating photosensitive detector array.

[0032] Figure 3 Schematic diagram of the structure of the composite dielectric grating photosensitive detector in the embodiment.

[0033] Figure 4FIG. 4 is a block diagram of an odd-even word line selection module for controlling the reset voltage time length.

[0034] Figure 5 FIG. 4 is a block diagram of an odd-even word line selection module for controlling the length of the waiting voltage time.

[0035] Figure 6 It is a schematic diagram of the composite dielectric grating photosensitive detector to achieve a single-frame spatial domain multi-exposure large dynamic range.

[0036] Figure 7 It is a processing block diagram of a composite dielectric grating photosensitive detector to realize a single-frame large dynamic range image. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0038] Embodiment 1:

[0039] This embodiment provides a method for realizing single-frame large dynamic imaging based on a composite dielectric grating photosensitive detector. The method is applied to a pixel array composed of composite dielectric grating photosensitive detectors, and the top control gates of the detectors in the same row are connected to form a word line WL, the drain ends of the detectors in the same column are connected to form a bit line BL, and the source ends are connected to form a source line SL; Figure 1 and Figure 2 As shown in the figure, when the detector is working, different word lines are sequentially applied with a negative bias voltage, a zero bias voltage and a slowly increasing ramp voltage under the control of the word line selection module to drive the detector of the row into reset mode, exposure mode and readout mode respectively. There is a fixed interval between different word lines. , the time lengths of the reset mode, exposure mode and readout mode of all row word lines are consistent; the method comprises:

[0040] Step 1, based on the odd-numbered row and even-numbered row word line selection modules, the odd-numbered row and even-numbered row of the detector are controlled to perform exposure of different time lengths respectively, so as to obtain long exposure image data and short exposure image data;

[0041] Step 2, interpolating the long exposure picture data and the short exposure picture data along the vertical word line direction to obtain the interpolated long exposure picture data and short exposure picture data;

[0042] Specifically, the Lanczos kernel function may be used to calculate the corresponding weight for each pixel in the long-exposure image data and the short-exposure image data, and the weights of all pixels and the interpolation values ​​may be multiplied to obtain the final value of the interpolation point.

[0043] Step 3: Fusing the interpolated long-exposure image data and short-exposure image data to obtain a single-frame large dynamic image.

[0044] Embodiment 2:

[0045] This embodiment provides a method for realizing single-frame large dynamic imaging based on a composite dielectric grating photosensitive detector. The method is applied to a pixel array composed of a composite dielectric grating photosensitive detector, such as Figure 1 and Figure 2 As shown in the figure, when the detector is working, different word lines are sequentially applied with a negative bias voltage, a zero bias voltage and a slowly increasing ramp voltage under the control of the word line selection module to drive the detector of the row into reset mode, exposure mode and readout mode respectively. There is a fixed interval between different word lines. , the time lengths of the reset mode, exposure mode and readout mode of all row word lines are consistent.

[0046] The composite dielectric gate photosensitive detector constituting the pixel array is a composite dielectric gate MOSFET photosensitive detector disclosed in patent WO2010 / 094233 A1, such as Figure 3 As shown, the composite dielectric grating photosensitive detector includes a MOS-C part and a MOSFET part, and the two parts are formed on the same P-type semiconductor substrate. Among them, the MOS-C part includes a first dielectric layer, a charge coupling layer, a second dielectric layer and a first control gate stacked in sequence on the P-type semiconductor substrate; the MOSFET part includes a first dielectric layer, a charge coupling layer, a second top dielectric layer and a second control gate stacked in sequence on the P-type semiconductor substrate, an N-type source region and an N-type drain region are provided on one side of the P-type semiconductor substrate and close to the bottom dielectric layer, and a threshold adjustment injection region is provided in the P-type semiconductor substrate and below the bottom dielectric layer.

[0047] like Figure 1 As shown, the pixel array is composed of composite dielectric gate photosensitive detectors. The top control gates of the detectors in the same row are connected to form a word line (WL), the drain ends of the detectors in the same column are connected to form a bit line (BL), and the source ends are connected to form a source line (SL). The substrates of all transistors are connected uniformly and connected to a fixed bias voltage of -3V. The word line direction of the array is horizontal, and the odd-numbered row word lines correspond one-to-one to the output ends of the odd-numbered row word line selection module, and the even-numbered row word lines correspond one-to-one to the output ends of the even-numbered row word line selection module. If the word line is selected under logic control, the readout tubes located in the entire row of word lines will be turned on at the same time.

[0048] Before exposure begins, a reset voltage Vs is applied to the pixel array substrate of odd or even rows, so that the reset voltage is less than the depletion voltage Vb of the carrier collection region. During this period, no carriers are generated. The reset voltage application time for odd and even rows is respectively and , assuming , after the reset time ends, the carriers in the pixel are cleared.

[0049] During the exposure process of the array, a negative bias pulse Vb is applied to the substrate; when reading the selected pixel unit, the substrate is grounded, and a positive bias Vgr is applied to the word line of the row where the pixel unit to be read is located. In this process, the reading and waiting voltage application time of the odd and even rows are set to be the same, respectively. , and , The total length of the word line gating module in the function gating mode remains unchanged, and the length of the odd and even rows is the same. ,like Figure 2 As shown, the exposure times for odd and even rows are:

[0050]

[0051]

[0052] Depend on It can be seen that , odd-numbered rows are set as long-exposure rows for signal readout, and even-numbered rows are set as short-exposure rows for signal readout.

[0053] like Figure 5 As shown in FIG. 1 , the total length of the timing of the word line gating module in the function gating mode remains unchanged. The word line gating module increases the length of the waiting voltage in odd or even rows, assuming that Before exposure begins, a reset voltage Vs is applied to the pixel array substrate of odd or even rows, so that the reset voltage is less than the depletion voltage Vb of the carrier collection region. During this period, no carriers are generated. The reset voltage application time for odd and even rows is respectively and , assuming When the reset time is over, the carriers in the pixel are cleared. When reading the selected pixel unit, the substrate is grounded, and a positive bias voltage Vgr is applied to the word line of the row where the pixel unit to be read is located. In this process, the reading time of odd and even rows is set to be the same. , waiting time for voltage to be applied , then the exposure times of odd and even rows are:

[0054]

[0055]

[0056] Depend on It can be seen that , odd-numbered rows are set as long-exposure rows for signal readout, and even-numbered rows are set as short-exposure rows for signal readout.

[0057] like Figure 6 As shown in the figure, the single frame image after readout contains two parts: long exposure and short exposure. The circuit noise, dark level noise, exposure time and full well charge are , , , , the minimum current and maximum current of the long exposure part are and The minimum and maximum currents of the short exposure part are 0 and , then the dynamic range of a single pixel is:

[0058]

[0059] Compared with the non-single-frame large dynamic mode, it is increased by a times, so the dynamic range of the image in the single-frame large dynamic mode is increased by a times.

[0060] like Figure 7 As shown, after the odd-row detector and the even-row detector have different exposure times through two word line selection modules, the image data of the odd-row and even-row are read respectively, which is recorded as and ,in and are the total number of rows and columns of the detector respectively. The image data of odd and even rows are interpolated by Lanczos based on 8×8 pixel neighborhood, and its kernel function is:

[0061] ,

[0062] First, determine the position of the interpolation point, take the interpolation point as the center, and or Take a window and use the Lanczos kernel function to calculate the weight of each pixel in the window. Usually the kernel function is . Multiply the weights of all pixels in the window by the interpolation value to get the final value of the interpolation point.

[0063] Interpolated image odd-row matrix and the matrix with even rows ;in, Indicates the odd-numbered lines of image data read. Indicates the even-numbered rows of image data read. and the matrix with even rows Do it separately Transformation, get the top image of the Laplace pyramid, respectively and , take a size of Neighborhood, calculate and The mean gradient of each pixel in is calculated with each pixel as the center and the neighborhood size is ,Since the gradient reflects the detail changes and texture changes in the ,image, the larger the gradient is, the higher the image fusion clarity is . The grayscale value of the pixel at can be represented by the following pixel, the fused image:

[0064] ,

[0065] Leveraging the Camera Response Model , is the grayscale value of the pixel, is the ambient radiance, is the exposure time, are the camera lens parameters, is the camera response function, and the image after long and short exposure is corrected according to the calibrated camera response curve If there is no prior data on the camera response function, ,in , is an odd-numbered matrix and the matrix with even rows The weighted average of .

[0066] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.

[0067] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for realizing single-frame large dynamic imaging based on a composite dielectric grating photosensitive detector, characterized in that: The method is applied to a pixel array composed of composite dielectric grating photosensitive detectors, wherein the top control gates of the detectors in the same row are connected to form a word line WL, the drain ends of the detectors in the same column are connected to form a bit line BL, and the source ends are connected to form a source line SL; the method comprises: Step 1, based on the odd-numbered row and even-numbered row word line selection modules, the odd-numbered row and even-numbered row of the detector are controlled to perform exposure of different time lengths respectively, so as to obtain long exposure image data and short exposure image data; Step 2, interpolating the long exposure picture data and the short exposure picture data along the vertical word line direction to obtain the interpolated long exposure picture data and short exposure picture data; Step 3, fusing the interpolated long exposure image data and short exposure image data to obtain a single-frame large dynamic image; In step 2, the Lanczos kernel function is used to calculate the corresponding weight for each pixel in the long exposure image data and the short exposure image data, and the weight of all pixels is multiplied by the interpolation value to obtain the final value of the interpolation point. The step 2 comprises: Step 2.1, read the odd and even rows of image data, recorded as and ,in and are the total number of rows and columns of detectors, respectively; Step 2.2: The odd-numbered and even-numbered image data are converted into Lanczos interpolation of the pixel neighborhood to obtain the interpolated image odd-numbered row matrix and the matrix with even rows ; Step 2.3, change the odd-numbered rows of the matrix and the matrix with even rows Do it separately Transformation, the top-level images of the Laplace pyramid are recorded as and ; Step 2.4, take a size of The neighborhood is centered on each pixel and has a neighborhood size of Calculate separately and The gradient mean corresponding to each pixel in is taken as the gray value of the pixel; Step 2.5, get the fused image based on the gray value of each pixel, and compare and The larger value is used as the value of the corresponding pixel; The Lanczos interpolation kernel function is: in, represents the smoothing coefficient; The image odd-numbered row matrix after interpolation in step 2.2 and the matrix with even rows for: in, Indicates the odd-numbered lines of image data read. Indicates the even-numbered lines of image data read.

2. The method according to claim 1, characterized in that In the pixel array, odd-numbered word lines correspond to output ends of odd-numbered word line gating modules one by one, and even-numbered word lines correspond to output ends of even-numbered word line gating modules one by one; Set different row word lines to apply a negative bias voltage, a zero bias voltage and a ramp voltage in sequence under the control of the word line gating module to drive the corresponding row detector to enter the reset mode, exposure mode and readout mode respectively. The timing cycle T of the word line gating modules of different rows in the functional gating mode is the same, and the starting time of the timing cycle T of different row word lines has a fixed interval .

3. The method according to claim 2, characterized in that The timing cycle T includes reset time, exposure time, readout time and waiting time for voltage application; the method controls the waiting time for voltage application or reset time to be different through odd-numbered word lines and odd-numbered word line selection modules so as to make the exposure time different.

4. The method according to claim 3, characterized in that The composite dielectric grating photosensitive detector includes a MOS-C part and a MOSFET part, and the two parts are formed on the same P-type semiconductor substrate; wherein the MOS-C part includes a first dielectric layer, a charge coupling layer, a second dielectric layer and a first control gate stacked in sequence above the P-type semiconductor substrate; the MOSFET part includes a first dielectric layer, a charge coupling layer, a second top dielectric layer and a second control gate stacked in sequence above the P-type semiconductor substrate, an N-type source region and an N-type drain region are provided in the P-type semiconductor substrate and on one side close to the bottom dielectric layer, and a threshold adjustment injection region is provided in the P-type semiconductor substrate and below the bottom dielectric layer.

Images