Image sensor single event transient bright spot feature online extraction method based on image processing

Abstract

The present application relates to a kind of image sensor single particle transient bright spot feature online extraction method based on image processing, the device involved in this method includes power supply, CMOS image sensor, image sensor driving board, frame capture video collector, computer, CMOS image sensor is installed on image sensor driving board and is connected with frame capture video collector by Camera link line, frame capture video collector is connected with computer by USB.Computer is used in the process of proton irradiation CMOS image sensor, including image Fourier transform, threshold segmentation etc., image processing algorithm is carried out to the feature extraction of each image with single particle transient bright spot collected by frame capture video collector, and the output result includes the bright spot area, perimeter, maximum gray value, secondary gray value, minimum gray value, average gray value, gray value standard deviation and barycenter position information of single particle transient bright spot.The method does not need to save image to local, greatly saves storage space, can be used for longer time bright spot feature analysis, and bright spot feature can be output simultaneously with image acquisition to facilitate research.

Description

Technical Field

[0001] This invention relates to the field of image sensor performance parameter detection, and specifically to an online extraction method for single-particle transient bright spots in image sensors based on image processing. Background Technology

[0002] CMOS image sensors, with their advantages of high integration, low power consumption, and high dynamic range, have become the main optoelectronic imaging devices in aerospace missions. However, due to the influence of high-energy particles in the space radiation environment, transient bright spots appear in images captured by CMOS image sensors. These bright spots only appear at the same location in one frame and disappear in the next. To understand the characteristics of these transient bright spots, it is necessary to conduct accelerator irradiation experiments on the ground, using different energies and different particles to carry out online irradiation experiments on CMOS image sensors, and to extract and analyze the features of the transient bright spots in the acquired images.

[0003] Patent CN108401151A only collects images of single-event effect experiments to a computer without analyzing the characteristics of the single particles, requiring a large amount of storage space. Furthermore, during irradiation, gaps exist in the beam, resulting in images without transient bright spots remaining in the saved images. Patent CN103675546A uses image processing techniques such as global thresholding to process images containing single-event transients to obtain the characteristics of the transient bright spots. However, the grayscale value and shape of the transient bright spots are related to the type and energy of the incident particles. Without prior knowledge, using a single thresholding value is not applicable to every incident condition, requiring repeated parameter adjustments. This invention processes images acquired by a frame-capturing video capture device online, eliminating the need for saving storage space and enabling long-term online analysis of the characteristics of single-event transient bright spots from image sensors during irradiation. Meanwhile, this invention improves the single-particle transient bright spot extraction method by using a Gaussian difference filter composed of two Gaussian filter kernels and local threshold segmentation to obtain the bright spot region. It can extract the transient bright spot features caused by various particles without prior knowledge and has universal applicability. Summary of the Invention

[0004] The purpose of this invention is to provide an online method for extracting single-particle transient bright spot features from an image sensor based on image processing. The method involves a power supply, a CMOS image sensor, an image sensor driver board, a frame-capturing video acquisition device, and a computer. The CMOS image sensor is mounted on the image sensor driver board and connected to the frame-capturing video acquisition device via a Camera Link cable. The frame-capturing video acquisition device is connected to the computer via USB. Finally, the power supply is turned on to ensure the image sensor driver board is functioning normally. During proton irradiation of the CMOS image sensor, image processing algorithms, including image Fourier transform and threshold segmentation, are used on the computer to extract features from each image with a single-particle transient bright spot acquired by the frame-capturing video acquisition device. The output includes information such as the bright spot area, perimeter, maximum gray value, second-largest gray value, minimum gray value, average gray value, gray value standard deviation, and centroid position. This method eliminates the need to save images locally, greatly saving storage space, and can be used for long-term bright spot feature analysis. Furthermore, it can output bright spot features simultaneously with image acquisition for research purposes.

[0005] The present invention discloses an online image sensor single-particle transient bright spot feature extraction method based on image processing. The method involves a device comprising a CMOS image sensor (1), a power supply (2), an image sensor driver board (3), a frame-capturing video acquisition unit (4), and a computer (5). The CMOS image sensor (1) and the image sensor driver board (3) are connected to the power supply (2). One end of the frame-capturing video acquisition unit (4) is connected to the image sensor driver board (3), and the other end is connected to the computer (5). The specific operation is performed according to the following steps:

[0006] a. Before irradiation, fix the CMOS image sensor (1) onto the image sensor driver board (3), connect the output port of the image sensor driver board (3) to the input port of the frame capture video collector (4) using the CameraLink cable, connect the frame capture video collector (4) to the computer (5) via the USB port, turn on the power (2), fix the image sensor driver board (3) to the center of the radiation source beam, and keep the subsequent process free from stray light interference.

[0007] b. Configure the frame capture video collector (4) in the computer (5) to single frame acquisition mode, calculate and record the standard deviation 1 of the image captured by the frame capture video collector before the irradiation begins;

[0008] c. Create two Gaussian filters with standard deviations of 0.1 and 3 respectively in the computer (5). The width and height of the filters are the same as the images captured by the frame capture video collector (4). Subtract the Gaussian filter with a coefficient of 3 from the Gaussian filter with a coefficient of 0.1 to obtain the Gaussian difference filter.

[0009] d. After irradiation begins, set the frame capture video collector (4) in the computer (5) to continuous acquisition mode;

[0010] e. Calculate the standard deviation 2 of the image captured by the frame capture video collector (4). If the difference between the standard deviation 2 and the standard deviation 1 is greater than 20, it is determined that there is a single-particle transient bright spot in the image. Continue to execute the next step in sequence. If there is no single-particle transient bright spot in the image, execute step e again after the frame capture video collector (4) captures a new image.

[0011] f. Perform a Fast Fourier Transform on the image containing the single-particle transient bright spot in step e to obtain its frequency domain image;

[0012] g. Perform frequency domain convolution operation on the frequency domain image obtained in step f and the Gaussian difference filter obtained in step c, and perform inverse Fourier transform on the convolution result to obtain the filtered image.

[0013] h. The filtered image from step g is processed using a local threshold segmentation algorithm to obtain regions with single-particle transient bright spots and regions without single-particle transient bright spots. The mean and standard deviation of the gray values ​​of the regions without single-particle transient bright spots are calculated.

[0014] i. Divide the region containing single-particle transient bright spots obtained in step h into connected components, and calculate the minimum bounding rectangle region for each unconnected region obtained after the division.

[0015] j. Use the sum of the mean gray value obtained in step h and three times the standard deviation 3 as the threshold to perform threshold segmentation on each rectangular region obtained in step i, so as to obtain the accurate transient bright spot region.

[0016] k. Perform grayscale value statistics on the transient bright spot regions obtained in step j, and filter out transient bright spot regions with overlap based on the grayscale value distribution;

[0017] l. For each region containing transient bright spots after filtering in step k, perform feature extraction to obtain features such as bright spot area, perimeter, maximum gray value, second largest gray value, minimum gray value, average gray value, standard deviation of gray value, and centroid position.

[0018] m. Save the bright spot area, perimeter, maximum gray value, second largest gray value, minimum gray value, average gray value, standard deviation of gray value and centroid position extracted in step l to the computer according to the category (5).

[0019] n. For the next frame image captured by the frame capture video capture device (5), the execution starts from step e.

[0020] This invention discloses an online image sensor single-particle transient bright spot feature extraction method based on image processing. Compared with typical single-particle transient bright spot feature extraction methods, this method does not require storing the image locally and can complete the analysis of single-particle transient bright spot features during the accelerator beam exit process. Furthermore, this method does not specify a uniform threshold segmentation value during image processing, but uses a bandpass filter and a local threshold segmentation algorithm. No parameter adjustments are required during extraction, making it universally applicable to determining the location and influence range of bright spots generated under different incident conditions. After locating the transient bright spot in the image, the grayscale and shape features of the transient bright spot can be output as needed. After analyzing and statistically processing a large number of images containing single-particle transient bright spots, the patterns of single-particle transient bright spots at that energy level can be obtained.

[0021] This invention discloses an online image sensor single-particle transient bright spot feature extraction method based on image processing, applicable to the extraction of single-particle transient bright spot features generated by image sensors with different particle incident spectroscopy. It is suitable for research institutions and research organizations that require analysis of single-particle transient features from image sensors. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the testing system of the present invention;

[0023] Figure 2 This is an image captured by the CMOS image sensor of this invention before the irradiation begins;

[0024] Figure 3 The image acquired by the CMOS image sensor of this invention includes a single-particle transient.

[0025] Figure 4 This is a frequency domain image of an image containing a single-particle transient in this invention;

[0026] Figure 5 This is the image after bandpass filtering according to the present invention;

[0027] Figure 6 This invention includes an image of transient bright spots and the minimum bounding rectangle region of the transient bright spots;

[0028] Figure 7 The image containing transient bright spots and the area affected by transient bright spots are selected by this invention;

[0029] Figure 8 This represents the maximum grayscale value in each transient bright spot output by this invention. Detailed Implementation

[0030] Example

[0031] The present invention discloses an online image sensor single-particle transient bright spot feature extraction method based on image processing. The method involves a device comprising a CMOS image sensor 1, a power supply 2, an image sensor driver board 3, a frame-capturing video acquisition unit 4, and a computer 5. The CMOS image sensor 1 and the image sensor driver board 3 are connected to the power supply 2. One end of the frame-capturing video acquisition unit 4 is connected to the image sensor driver board 3, and the other end is connected to the computer 5. The specific operation is performed according to the following steps:

[0032] a. Before irradiation, fix the CMOS image sensor 1 onto the image sensor driver board 3. Connect the output port of the image sensor driver board 3 to the input port of the frame capture video acquisition device 4 using a Camera Link cable. The frame capture video acquisition device 4 is connected to the computer 5 via a USB port. Turn on the power supply 2 and fix the device with the image sensor driver board 3 at the center of the radiation source beam, ensuring no stray light interference in the subsequent process. The image sensor used is a commercial global exposure image sensor with a resolution of 2048×2048 (see Appendix). Figure 1 ;

[0033] b. Configure the frame capture video collector 4 in computer 5 to single-frame acquisition mode, calculate and record the standard deviation 1 (dev1) of the images acquired by the collector before irradiation begins. In this embodiment, this value is 5.53. (See Appendix) Figure 2 ;

[0034] c. Create two Gaussian filters with standard deviations of 0.1 and 3 respectively in computer 5. The width and height of the filters are the same as the images captured by the frame capture video capture device 4. Subtract the Gaussian filter with a coefficient of 3 from the Gaussian filter with a coefficient of 0.1 to obtain the difference Gaussian filter.

[0035] d. After irradiation begins, set the frame capture video collector 4 in computer 5 to continuous acquisition mode;

[0036] e. Calculate the standard deviation 2 (dev2) of the image captured by the frame capture video capture device 4. If the difference between standard deviation 2 (dev2) and standard deviation 1 (dev1) is greater than 20, it is determined that the image contains a single-event transient bright spot, and the next step is continued. If the image does not contain a single-event transient bright spot, step e is re-executed after the frame capture video capture device 4 captures a new image. In this embodiment, the standard deviation 2 (dev2) is 87.66, which differs from the standard deviation 1 (dev1) by more than 20, so it is determined that the image contains a single-event transient bright spot, and steps f to n are continued. (See Appendix) Figure 3 ;

[0037] f. Perform a Fast Fourier Transform on the image containing the single-particle transient bright spot in step e to obtain the frequency domain image (see appendix). Figure 4 ;

[0038] g. Perform frequency domain convolution on the frequency domain image obtained in step f and the Gaussian difference filter obtained in step c. Perform inverse Fourier transform on the convolution result to obtain the filtered image (see appendix). Figure 5 ;

[0039] h. Apply local thresholding to the filtered image from step g to obtain the region containing single-particle transient bright spots and the region other than single-particle transient bright spots. Calculate the mean (Mean) and standard deviation 3 (dev3) of the gray values ​​of the region without single-particle transient bright spots. In this embodiment, the Mean value is 152.06 and the standard deviation 3 (dev3) value is 7.544.

[0040] i. Divide the region containing the single-particle transient bright spot obtained in step h into connected components. Calculate the minimum bounding rectangle region for each unconnected region obtained after segmentation (see Appendix). Figure 6 ;

[0041] j. Use the sum of the mean gray value (Mean) obtained in step h and three times the standard deviation 3 (dev3) as the threshold to perform threshold segmentation on each rectangular region obtained in step i, so as to obtain the accurate transient bright spot region. In this embodiment, the threshold is 174.692.

[0042] k. Perform grayscale value statistics on the transient bright spot regions obtained in step j, and filter out overlapping transient bright spot regions based on the grayscale value distribution. (See appendix) Figure 7 ;

[0043] 1. For each region containing transient bright spots after filtering in step k, perform feature extraction to obtain features such as bright spot area, perimeter, maximum gray value, second largest gray value, minimum gray value, average gray value, standard deviation of gray value, and centroid position; in this embodiment, only the maximum and minimum gray values ​​in transient bright spots are counted.

[0044] m. Save the maximum and minimum grayscale values ​​of the bright spots extracted in step k to the computer, see appendix. Figure 8 ;

[0045] n. For the next frame image captured by the frame capture video capture device (5), the execution can start directly from step e.

[0046] The above description is merely a specific embodiment of the present invention, but the present invention is not limited thereto. Any substitutions or additions that can be understood by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of the present invention.

Claims

1. A method for extracting single-particle transient bright spot features from an online image sensor based on image processing, characterized in that, The device involved in this method consists of a CMOS image sensor (1), a power supply (2), an image sensor driver board (3), a frame-capturing video acquisition unit (4), and a computer (5). The CMOS image sensor (1) and the image sensor driver board (3) are connected to the power supply (2). One end of the frame-capturing video acquisition unit (4) is connected to the image sensor driver board (3), and the other end of the frame-capturing video acquisition unit (4) is connected to the computer (5). The specific operation is carried out according to the following steps: a. Before irradiation, fix the CMOS image sensor (1) onto the image sensor driver board (3), connect the output port of the image sensor driver board (3) to the input end of the frame capture video collector (4) using the Camera Link cable, connect the frame capture video collector (4) to the computer (5) via the USB port, turn on the power (2), fix the image sensor driver board (3) to the center of the radiation source beam, and keep the subsequent process free from stray light interference. b. Configure the frame capture video collector (4) in the computer (5) to single frame acquisition mode, calculate and record the standard deviation 1 of the image captured by the frame capture video collector before the irradiation begins; c. Create two Gaussian filters with standard deviations of 0.1 and 3 respectively in the computer (5). The width and height of the filters are the same as the images captured by the frame capture video collector (4). Subtract the Gaussian filter with a coefficient of 3 from the Gaussian filter with a coefficient of 0.1 to obtain the Gaussian difference filter. d. After irradiation begins, set the frame capture video collector (4) in the computer (5) to continuous acquisition mode; e. Calculate the standard deviation 2 of the image captured by the frame capture video collector (4). If the difference between the standard deviation 2 and the standard deviation 1 is greater than 20, it is determined that there is a single-particle transient bright spot in the image. Continue to execute the next step in sequence. If there is no single-particle transient bright spot in the image, execute step e again after the frame capture video collector (4) captures a new image. f. Perform a Fast Fourier Transform on the image containing the single-particle transient bright spot in step e to obtain its frequency domain image; g. Perform frequency domain convolution operation on the frequency domain image obtained in step f and the Gaussian difference filter obtained in step c, and perform inverse Fourier transform on the convolution result to obtain the filtered image. h. The filtered image from step g is processed using a local threshold segmentation algorithm to obtain regions with single-particle transient bright spots and regions without single-particle transient bright spots. The mean and standard deviation of the gray values ​​of the regions without single-particle transient bright spots are calculated. i. Divide the region containing single-particle transient bright spots obtained in step h into connected components, and calculate the minimum bounding rectangle region for each unconnected region obtained after the division. j. Use the sum of the mean gray value obtained in step h and three times the standard deviation 3 as the threshold to perform threshold segmentation on each rectangular region obtained in step i, so as to obtain the accurate transient bright spot region. k. Perform grayscale value statistics on the transient bright spot regions obtained in step j, and filter out transient bright spot regions with overlap based on the grayscale value distribution; l. For each region containing transient bright spots after filtering in step k, perform feature extraction to obtain features such as bright spot area, perimeter, maximum gray value, second largest gray value, minimum gray value, average gray value, standard deviation of gray value, and centroid position. m. Save the bright spot area, perimeter, maximum gray value, second largest gray value, minimum gray value, average gray value, standard deviation of gray value and centroid position extracted in step l to the computer according to the category (5). n. For the next frame image captured by the frame capture video capture device (4), the execution starts from step e.

Images