A method for deducting digital domain background noise of a large array CMOS detector

Abstract

The application relates to a method for deducting the background noise of a large-array CMOS detector in the digital domain, in which, firstly, a dark field image L1 generated by the large-array CMOS detector is set, and the dark field image is aligned and blocked, the mean value of the blocked image is counted, and the image block mean value matrix corresponding to the dark field image L1 is obtained; the detector is set to normally image, and a photosensitive image L2 is obtained; a radiation calibration dark field image L0 is obtained through a detector radiation calibration method, and the radiation calibration dark field image data is blocked according to the same division mode as the dark field image data, the mean value of the blocked image is counted, and the image block mean value matrix corresponding to the radiation calibration dark field image L0 is obtained; the variances of the two image block mean value matrices are calculated respectively, and the sizes of the two variances are compared to determine whether the dark field image is effective. The method realizes accurate deduction of the background of a large-array CMOS detector with a large number of pixels, improves the adaptability in the imaging process of the detector, and improves the signal-to-noise ratio index of the image.

Description

Technical Field

[0001] This invention relates to the field of space remote sensing image preprocessing technology, and in particular to a method for subtracting the background noise in the digital domain of a large-area CMOS detector. Background Technology

[0002] In CCD camera applications, to effectively reduce image noise, correlated double sampling (CDS) is typically performed on the detector video signal to suppress detector reset noise. The CDS function is integrated into the video processor chip, leveraging the high performance of integrated circuits to implement the video signal CDS function.

[0003] Inside the video processor chip, the reset level signal strength and signal level signal strength of the detector video signal are acquired, ensuring that the time interval between two samplings is small enough to make the acquired noise voltages essentially consistent. Then, in the analog circuit, charge subtraction is performed to obtain the video signal after the reset noise is suppressed. Commonly used video processors include TDA9965 and LM98640.

[0004] In CMOS photodetector applications, the CMOS detector directly outputs digital signals. During data transmission, the digital signal is less susceptible to interference from external noise sources, thus improving image data quality. Each pixel in a CMOS detector has a corresponding charge transfer circuit and an analog-to-digital (AD) conversion unit, and noise reduction processing is also implemented internally. Because the circuit states and operating conditions of each pixel differ, the internal circuit noise of the detector, such as reset noise, quantization noise, readout noise, and thermal noise, is determined by the detector's performance. Different operating states of the detector affect the image data output quality.

[0005] Besides using CDS to suppress reset noise, when designing linear CCD or CMOS detector circuits, acquiring the dark pixels output by the detector, calculating the average value of dark pixels in the row image, and then subtracting the background noise from the row image data can effectively suppress power supply noise and background noise interference in the row image. However, in the design of imaging circuits for large-area CMOS detectors, the noise characteristics of each pixel are different, increasing the variability of large-area detectors across the entire field of view.

[0006] In large-area CMOS detectors, video signal processing circuitry is integrated within the detector itself. The video signal undergoes CDS / AD conversion within the detector before being output as a digital signal. However, due to the large pixel size of large-area CMOS detectors, pixel background inhomogeneity and background variations caused by thermal noise cannot be eliminated, leading to problems such as image inhomogeneity and increased noise, thus reducing the detector's imaging quality.

[0007] Currently, obtaining background image data for large-area CMOS detectors through ground-based radiometric calibration and performing background subtraction during imaging cannot suppress the effects of detector pixel thermal noise and background fluctuations, thus failing to achieve real-time, high-precision processing. Therefore, there is an urgent need to implement digital background subtraction processing for large-area CMOS detectors to improve the accuracy and adaptability of detector background subtraction. Summary of the Invention

[0008] To address the issues of inability to remove background non-uniformity and thermal noise-induced background variations, which lead to image non-uniformity and increased noise, and the inability to suppress the influence of detector pixel thermal noise and background fluctuations through ground radiation calibration, thus failing to achieve real-time and refined processing, this invention provides a method for subtracting digital domain background noise from a large-area CMOS detector.

[0009] To solve the above problems, the present invention adopts the following technical solution:

[0010] A method for subtracting the background noise in the digital domain of a large-area CMOS detector is disclosed. Before acquiring each frame of image, a dark-field image of the detector is pre-acquired. The acquisition process of the large-area CMOS detector and the process of subtracting the background noise from the image include the following steps:

[0011] Set a large-area CMOS detector to generate a dark field image L1. If the large-area CMOS detector does not have a dark field image output function, then use the image with the minimum exposure time as the dark field image L1. Then divide the dark field image data into blocks, calculate the mean of the block images, and obtain the image block mean matrix corresponding to the dark field image L1.

[0012] Set up a large-area CMOS detector for normal imaging to obtain photosensitive image L2;

[0013] The radiation calibration dark field image L0 is obtained by the detector radiation calibration method. The radiation calibration dark field image data is divided into blocks according to the same division method as the dark field image data. The mean value matrix of the image blocks corresponding to the radiation calibration dark field image L0 is obtained by statistically analyzing the mean value of the block images.

[0014] Calculate the variances of the image patch mean matrices for dark field image L1 and radiometrically calibrated dark field image L0, respectively, and denote them as δ1 and δ0.

[0015] When δ1 < 3δ0, the image data of dark field image L1 is determined to be valid, and dark field image L1 is subtracted from photosensitive image L2; when δ1 ≥ 3δ0, the image data of radiometric calibration dark field image L0 is determined to be valid, and radiometric calibration dark field image L0 is subtracted from photosensitive image L2.

[0016] This invention proposes a digital domain background noise reduction method for the application characteristics of large-area CMOS detectors. Before acquiring each frame of image from the large-area CMOS detector, a dark-field image of the detector is pre-acquired to obtain the background value of the detector pixels under the current operating conditions, and this background value is then subtracted from the image. When the ambient temperature or the detector's operating state changes, the acquired dark-field background image changes accordingly. The image after real-time background subtraction better shields the effects of changes in the detector's state. This method improves the adaptability of the detector during the imaging process and enhances the signal-to-noise ratio of the image.

[0017] Compared with existing CCD and CMOS detector background subtraction techniques, the present invention has the following advantages:

[0018] 1) Suitable for large-area CMOS detectors, processing more pixels and possessing portability;

[0019] 2) Applicable to both large-area CMOS detectors with dark-field image output function and those without dark-field image output function;

[0020] 3) Achieve accurate subtraction of the background of massive pixels in a large-area CMOS detector;

[0021] 4) The 3δ method was used to realize the function of interpreting the image background data under the minimum exposure time. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of a method for subtracting the digital domain noise floor of a large-area CMOS detector according to an embodiment of the present invention;

[0023] Figure 2 This is a timing diagram for the background subtraction drive of a large-area CMOS detector in an embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram illustrating the control of detector data storage at the same address using the address mask of the DDR3-IP core in an embodiment of the present invention.

[0025] Figure 4 The image is a visible light camera image obtained by using the background noise reduction method described in the embodiments of the present invention. Detailed Implementation

[0026] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings and preferred embodiments.

[0027] This invention provides a digital noise floor reduction method to improve image quality of a large-area CMOS detector. The method first acquires a noise floor image of the large-area CMOS detector, then acquires a photosensitive image of the detector, and subtracts the detector noise floor in the digital domain to obtain a noise floor-suppressed image. During image acquisition, the time interval between two sampling steps must be sufficiently small to ensure that the acquired noise floor voltages are essentially consistent.

[0028] Specifically, such as Figure 1 As shown, the acquisition process of the large-area CMOS detector and the process of removing background noise from the image include the following steps:

[0029] The detector's dark-field image characterizes the detector's pixel background and circuit noise characteristics under current parameter settings. Under constant internal and external conditions, the grayscale values ​​of the detector's dark-field image fluctuate within a small range over a short period, representing the detector's background noise level under current environmental conditions. For some large-area CMOS detectors that cannot output dark-field images in real time, the minimum exposure time image is used instead. According to experimental measurements, a minimum exposure time t0 less than 100 μs, an exposure time setting t1 greater than 1 ms during normal imaging, and t1 > 50t0 ensure that the minimum exposure image data can be used as a dark-field image for background subtraction processing.

[0030] According to the detector design manual, the detector driving timing is designed. First, the large-area CMOS detector is set to generate a dark field image L1. The dark field image data L1 is divided into blocks, and the mean value of the block images is calculated to obtain the image block mean matrix corresponding to the dark field image L1, which is used as the image background subtraction parameter. Then, the large-area CMOS detector is set to normal imaging to obtain the photosensitive image L2.

[0031] The radiation calibration dark field image L0 is obtained by the detector radiation calibration method, and the dark field image data is stored in the circuit. The radiation calibration dark field image data L0 is divided into blocks according to the same division method as the dark field image data L1.

[0032] Next, the variance δ1 of the image patch mean matrix (denoted as V1) of the dark-field image L1 and the variance δ0 of the image patch mean matrix (denoted as V0) of the radiometrically calibrated dark-field image L0 are calculated and compared. Based on experience, when δ1 < 3δ0, the image data of the dark-field image L1 is considered valid, and the dark-field image data L1 is subtracted from the photosensitive image L2; when δ1 ≥ 3δ0, the image data of the dark-field image L1 is considered invalid, and the image data of the radiometrically calibrated dark-field image L0 is valid, and the radiometrically calibrated dark-field image L0 is subtracted from the photosensitive image L2 (this is referred to as the 3δ method). After background subtraction in the photosensitive image L2, the subtracted image is obtained, which is the image with suppressed background.

[0033] The workflow of the large-area CMOS image detector in this embodiment is as follows: detector reset—detector operating parameter loading—post-capture image data training—dark field (minimum exposure time) setting—acquiring dark field image—normal exposure time setting—acquiring photosensitive image.

[0034] Among them, "post-camera image data training" is the training graphic of the detector output image. The FPGA is used to perform delayed sampling of the input digital signal to obtain the optimal sampling position, so as to ensure that the detector output image data can be correctly acquired.

[0035] Large-area CMOS detector background subtraction drive timing as follows Figure 2 As shown. The main flow is referenced to the time axis. D0 represents the charge transfer time after the detector exposure ends, t0 represents the detector dark field image setting time or the minimum settable exposure time, and t1 represents the exposure time set during normal detector imaging. The driving timing of each stage is based on the detector design manual.

[0036] The large-area CMOS detector outputs image data using a multi-channel synchronous output method. In this project, an FPGA is used to receive the image data. Inside the FPGA, an image data storage structure is designed based on the CMOS detector's data output structure; similar methods can be used for the data storage structures of different detector models.

[0037] In this embodiment, the address mask of the DDR3-IP core is used inside the FPGA to implement the same address data storage control for the detector, achieving single-address storage of the same image data for dark field image L1 and photosensitive image L2. The following explanation uses a certain type of detector as an example to illustrate the detector data access control. Figure 3 As shown.

[0038] The detector synchronously outputs 80 channels of image data, which exhibit phase differences. After image data acquisition is completed within the FPGA, phase synchronization is achieved through a RAM level 1 cache design.

[0039] The image data is reassembled to obtain 512-bit wide image data. Bits 479 to 0 correspond to 40 channels of 12-bit quantized image data, while bits 512 to 480 are used as data extension bits and filled with 0s. These two sets of 512-bit image data are processed in a time-division multiplexing manner and stored in external SDRAM using the FPGA's internal DDR3-IP core. The program design employs a maximum rate matching principle, ensuring that the detector's image data output rate can guarantee that the SDRAM reaches its maximum storage rate, thereby increasing the detector's frame rate and ensuring better adaptability of the dark-field image L1.

[0040] The DDR3-SDRAM IP core user interface data bus width is set to 1024 bits. Utilizing the user interface mask provided by the IP core, the dark field image L1 is stored in the high 512 bits of the same address space, and the photosensitive image L2 is stored in the low 512 bits. The dark field image L1 and the photosensitive image L2 share the same address space, facilitating background digital subtraction processing.

[0041] During the process of storing image data in SDRAM, the sensor image data is divided into blocks and its features are statistically analyzed. Block division can employ various methods, such as row extraction (sampling points from rows of the image as independent blocks), column extraction (sampling points from columns of the image as independent blocks), and block segmentation (fixed-size blocks, such as 16×16 pixels), with the appropriate method chosen based on the data structure. The purpose of image block division is to statistically analyze the noise characteristics within each block; the continuity of the image content within a block is not required. After calculating the mean of the segmented images to form an image block mean matrix, the variance of the image block mean matrix is ​​calculated, serving as the criterion for determining whether a dark-field image can be used for background subtraction.

[0042] After a frame of image data is stored, 1024 bits of "dark field + photosensitive" image are read into the L2 cache RAM according to the SDRAM storage address. The validity of the dark field image is determined based on the variance of the calculated image block mean matrix. The 3δ method is used for the determination, as described above, and will not be repeated here.

[0043] Background subtraction calculation is performed on the photosensitive image L2. After the calculation is completed, data transmission is started. The image output rate is controlled by the L2 cache RAM. After the transmission is completed, the acquisition of the next frame of dark field image and photosensitive image begins.

[0044] The method for subtracting the digital domain background noise of a large-area CMOS detector proposed in this embodiment achieves accurate subtraction of the pixel background of the large-area CMOS detector by providing dark field data under the same operating conditions for each pixel, thereby obtaining a detector image with good uniformity, reducing the detector background noise, and improving the imaging quality.

[0045] Compared with existing CCD and CMOS detector background subtraction techniques, this method has the following advantages:

[0046] 1) Suitable for large-area CMOS detectors, processing more pixels and possessing portability;

[0047] 2) Applicable to both large-area CMOS detectors with dark-field image output function and those without dark-field image output function;

[0048] 3) Achieve accurate subtraction of the background of massive pixels in a large-area CMOS detector;

[0049] 4) The 3δ method was used to realize the function of interpreting the image background data under the minimum exposure time.

[0050] The following is a specific example to further illustrate the technical solution of the present invention.

[0051] The image output sequence of a large-area CMOS detector used in a certain camera model is described as follows: Detector image data is output synchronously through 82 data channels, with each channel outputting 252 columns of detector pixels. That is, channel 1 outputs pixels 1-252, channel 2 outputs pixels 253-504, and so on. The detector outputs image data row by row, for a total output of 10240×10240 image data.

[0052] When framing image data, each 41-channel image data is composed of 512-bit wide image data, of which the lower 492 bits are valid data and the higher bits are filled with 0; the 82 channels generate two sets of 512-bit wide image data, denoted as data1-i and data2-i, i∈[1,252].

[0053] Two sets of image data, data1-i and data2-i, are stored in DDR3-SDRAM using a time-sharing storage method. The parameter settings of the DDR3-IP core are based on the image data input frequency and the bus width of the external SDRAM.

[0054] During image data storage, the average value of image data blocks is statistically analyzed. In this application, the blocks are segmented using a row sampling and column-based method: First, channels 1 to 32 of the 41-channel image data are selected, and the average value of the input image data is calculated; after statistical analysis of each row of image data, 252 average data points are obtained, resulting in 64 × 252 average data points for every 64 rows of images. Then, the column-wise average value is calculated to obtain 1 × 252 average data points for each image block. Using this method, a 160 × 252 image average data matrix is ​​ultimately obtained, which characterizes the average distribution of the detector's dark-field image within a local region.

[0055] The above block segmentation process uses a whole-row sampling method, which can also characterize the trend of image mean change. For dark-field images of the detector, it can achieve the same effect as "obtaining the image mean by continuous sampling".

[0056] After obtaining the mean matrix of the image patches, the variance δ of the mean matrix is ​​calculated using the following formula:

[0057]

[0058] Among them, V ij Let be the value of the element in the i-th row and j-th column of the image mean matrix, where n is the maximum number of rows in the image mean data matrix (n = 160), and m is the maximum number of columns in the image mean data matrix (m = 252). This is the global mean.

[0059] The calculation result is compared with the calculation result of the radiometric calibration dark-field image data to determine whether the dark-field image is usable. In this example, the calculated result of the real-time variance δ is <50, and the calculated result of the variance of the radiometric calibration dark-field data is 25.4. Therefore, this dark-field image can be used as a dark-field reference image for image background subtraction. The real-time visible light camera image after dark-field subtraction is as follows: Figure 4 As shown.

[0060] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0061] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for subtracting the digital domain noise floor of a large-area CMOS detector, characterized in that, Before acquiring each frame of image, the large-area CMOS detector pre-acquires a dark-field image of the detector. The acquisition process of the large-area CMOS detector and the process of subtracting background noise from the image include the following steps: Set a large-area CMOS detector to generate a dark field image L1. If the large-area CMOS detector does not have a dark field image output function, then use the image with the minimum exposure time as the dark field image L1. Then divide the dark field image data into blocks, calculate the mean of the block images, and obtain the image block mean matrix corresponding to the dark field image L1. Set up a large-area CMOS detector for normal imaging to obtain photosensitive image L2; The radiation calibration dark field image L0 is obtained by the detector radiation calibration method. The radiation calibration dark field image data is divided into blocks according to the same division method as the dark field image data. The mean value matrix of the image blocks corresponding to the radiation calibration dark field image L0 is obtained by statistically analyzing the mean value of the block images. Calculate the variances of the image patch mean matrices for dark field image L1 and radiometrically calibrated dark field image L0, respectively, and denote them as δ1 and δ0. When δ1 < 3δ0, the image data of dark field image L1 is determined to be valid, and dark field image L1 is subtracted from photosensitive image L2; when δ1 ≥ 3δ0, the image data of radiometric calibration dark field image L0 is determined to be valid, and radiometric calibration dark field image L0 is subtracted from photosensitive image L2.

2. The method for subtracting the digital domain noise floor of a large-area CMOS detector according to claim 1, characterized in that, The workflow of a large-area CMOS detector is as follows: reset – loading working parameters – training image data after imaging – setting minimum exposure time – acquiring dark-field images – setting normal exposure time – acquiring light-sensitive images.

3. A method for subtracting the digital domain noise floor of a large-area CMOS detector according to claim 1 or 2, characterized in that, The large-area CMOS detector outputs image data to the FPGA using a multi-channel synchronous output method. Inside the FPGA, the address mask of the DDR3-IP core is used to control the storage of data at the same address as the detector. The specific storage control method is as follows: After image data acquisition is completed inside the FPGA, phase synchronization is achieved through RAM level 1 cache design; The image data is reassembled to obtain image data with the corresponding bit width. The two sets of image data with the corresponding bit width are processed in a time-division manner and stored in external SDRAM using the FPGA's internal DDR3-IP core. Configure the user interface data bus width of the DDR3-SDRAM IP core, and use the user interface mask provided by the IP core to store the dark field image L1 in the high storage space and the photosensitive image L2 in the low storage space at the same address.

4. A method for subtracting the digital domain noise floor of a large-area CMOS detector according to claim 1 or 2, characterized in that, When dividing dark field image data or radiometric calibration dark field image data into blocks, any of the following block division methods can be used: row extraction, column extraction, and block partitioning.

5. A method for subtracting the digital domain noise floor of a large-area CMOS detector according to claim 1 or 2, characterized in that, The formula for calculating the variance of the image patch mean matrix is ​​as follows: Among them, V ij Let be the value of the element in the i-th row and j-th column of the image patch mean data matrix, where n is the maximum row size of the image patch mean matrix, and m is the maximum column size of the image patch mean matrix. This is the global mean.

Images