A method for calibrating and eliminating smear in interline transfer CCD cameras

Abstract

A method for calibrating and eliminating smear in an inter-line transfer CCD camera includes the following steps: S1: Based on the formation principles of smear and the original image, the relationship between the smear s(i,j) generated during charge transfer for each pixel and the effective image g(i,j) after smear elimination is derived. Furthermore, based on the smear transfer rule, the relationship between the smear s(i,j) and the original image m(i,j) during calibration is derived. Ultimately, the correlation between the uniform smear #imgabs0# experienced by pixels in the jth column and the accumulated value #imgabs1# of the jth column of the original image is derived. S2: Calibrate the camera, capture images with the camera, and fit multiple scatter plots obtained at different exposure times t0. The fitted relationship between the smear #imgabs2# and the accumulated value #imgabs3# of the jth column of the original image is obtained. S3: Eliminate smear during actual camera capture. This method requires only a single frame of image, resulting in a fast and simple elimination process. After a single calibration, it can be applied to capture images at various exposure times.

Description

Technical Field

[0001] The present invention relates to the technical field of photoelectric measurement, and in particular to a method for calibrating and eliminating smear of an inter-line transfer CCD camera. Background Art

[0002] As one of the most commonly used scientific-grade cameras, CCD (Charge Coupled Device) cameras offer significant advantages, including low noise, high sensitivity, a wide dynamic range, and high spatial and temporal resolution. CCDs can be categorized by their transfer method: frame transfer, interline transfer, and frame-interline transfer CCDs. Compared to frame transfer and frame-interline transfer CCDs, interline transfer CCDs do not require a signal charge storage unit and offer a more balanced performance in terms of manufacturing process, cost, image quality, and readout time, making them an ideal choice for compact and lightweight cameras.

[0003] Smear is a type of noise unique to CCD cameras. It is caused by the mixing of noise charges introduced during the transfer process and the signal charges. It generally appears as vertically drawn linear or banded noise on the captured image. In an interline transfer CCD camera, the photodiode 2 is used for photoelectric conversion. The exposure device accumulates the charge related to the light flux. After the exposure is completed, the charge is transferred to the corresponding vertical CCD 3. The vertical CCD 3 then transfers the charge downward to the horizontal CCD 4 and is output through the amplifier 5. A photodiode 2 and its corresponding vertical CCD 3 constitute a pixel 1. The vertical CCD 3 is used to transfer charge in a light-shielded environment, such as Figure 1 As shown in the figure, the primary cause of smear in interline transfer CCD cameras is incomplete light shielding of the vertical CCD3, which allows a significant amount of noise charge to leak into the signal charge transfer process. This incomplete light shielding can be caused by factors such as the limited transmittance of the relevant light shielding components in infrared and other wavelength bands, or hardware defects in the components.

[0004] Existing pixel removal methods primarily include: 1. Hardware optimization: For example, by implementing photoelectric blocking layers for pixels and rapidly discharging charge during non-integration time, extremely high switching speeds are achieved, thereby minimizing smear. 2. Algorithmic processing: For example, by binarizing dark pixel areas, replacing dark reference lines with virtual reference lines, subtracting adjacent frames in edge-mode operation, and subtracting exposed and unexposed images. Compared to hardware optimization, algorithmic processing offers advantages in feasibility, operability, and cost. However, existing algorithms often rely on processing multiple frames or locating dark pixels, which makes rapid and real-time noise removal challenging. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention proposes a method for calibrating and eliminating the smear of an inter-line transfer CCD camera. The specific technical solution is as follows:

[0006] A method for calibrating and eliminating smear of an interline transfer CCD camera comprises the following steps:

[0007] S1: Based on the formation principle of smear and original image, the relationship between the smear s(i,j) generated by the charge transfer of each pixel and the effective image g(i,j) after eliminating the smear is derived. The relationship between the smear s(i,j) and the original image m(i,j) at the time of calibration is derived based on the smear transfer rule. Finally, the uniform smear of the pixels in the jth column is derived. and the accumulated value of the jth column of the original image the relevant relationship;

[0008] S2: Calibrate the camera, use the camera to shoot, obtain multiple scatter plots at different exposure times t0, and fit them to get the smear and the accumulated value of the jth column of the original image The specific relationship of

[0009] S3: Eliminate the trailing image when shooting with the camera.

[0010] Specifically, the specific steps of step S1 are:

[0011] S11. The specific steps for obtaining the relationship between s(i,j) and g(i,j) are as follows:

[0012] The expression of the smear s(i,j) is

[0013] s(i,j)=η*k*t1*r(i,j) (1)

[0014] The expression of the effective image g(i,j) is

[0015] g(i,j)=k*t0*r(i,j)+b (2)

[0016] According to (1) and (2), the relationship between s(i,j) and g(i,j) is

[0017]

[0018] Where k is the camera's photoelectric conversion parameter; t0 is the exposure time; t1 is the charge transfer time per pixel; r(i,j) is the luminous flux per pixel; b is the image background noise caused by dark current; η is the ratio parameter between g(i,j) and s(i,j);

[0019] S12. The specific steps for obtaining the relationship between s(i,j) and m(i,j) are as follows:

[0020] Assume that the charge generated by the light-sensitive pixels in the jth column, i.e., the effective image of the jth column, is [g1, g2, g3, ..., gN-2 ,g N-1 ,g N ], where N is the number of pixels in a column; the corresponding smear noise charge generated by the charge transfer of each pixel is [s1,s2,s3,...,s N-2 ,s N-1 ,s N ]; In the first frame of charge transfer, the output charge of this column is

[0021] Starting from the second frame, the output charge of this column is

[0022] The smear effect on the original image is as follows: the sum of the smears that each pixel in this column experiences during the transfer is added to the original photosensitive charge of each pixel, and we can get (4):

[0023]

[0024] S13, derive the uniform smear of the pixels in the jth column and the accumulated value of the jth column of the original image The correlation relationship.

[0025] Specifically, the specific steps of step S13 are as follows:

[0026] S131. Combining (3) and (4), we can get (5):

[0027]

[0028] S132. Performing a single-column accumulation operation on both sides of equation (5) yields (6):

[0029]

[0030] S133. Simplify (6) to (7):

[0031]

[0032] S134. Let the slope of the linear relationship of formula (7) be K, and further express K as K(t0); and let the x-axis intercept be B, which is independent of t0, then the following formula is obtained:

[0033]

[0034] B=-N*b (9)

[0035]

[0036] Specifically, step S2 is as follows:

[0037] S21. Use the camera to shoot, exposure time t0, obtain a frame of original image m(i,j) in the calibration, and calculate the cumulative value of each column of the original image

[0038] S22. Measure the total amount of smear that each pixel in the image experiences during transfer.

[0039] S23: Take photos at multiple exposure times and repeat S21 to S22 to obtain the corresponding values ​​of different t0. and

[0040] S24: is the independent variable, As the dependent variable, draw multiple scatter plots at different t0;

[0041] S25: Perform linear fitting on multiple scatter plots at different t0 to obtain the slope K(t0) and the x-axis intercept B;

[0042] S26: Take t0 as the independent variable and K(t0) as the dependent variable, make a scatter plot, use formula (8) as the relationship to perform inverse proportional fitting, obtain the parameter η*t1, and determine and The specific relationship of .

[0043] Specifically, step S22 is as follows:

[0044] S221: Manually select the dark reference row in the dark area of ​​m(i,j);

[0045] S222: Averaging the dark reference lines to obtain image smear

[0046] Specifically, in step S221 , the dark reference lines are located at lines 10 to 100 at the upper and lower edges of the image.

[0047] Specifically, the fitting method in step S25 is the least squares method.

[0048] Specifically, step S3 is as follows:

[0049] S31: Use the camera to shoot, knowing the exposure time T0 and the original image M(i, j) corresponding to the actual shot, calculate according to step S2

[0050] S32: Substitute T0 into (8) with known η*t1 to obtain the corresponding K(T0);

[0051] S33: Substitute the calculated K(T0) and the known B in S25 into (10) to obtain the smear and The specific relationship between Find the value of each column

[0052] S34: Subtract the number of columns from the M(i,j) formula corresponding to each pixel. Then the image G(i,j) with the smear eliminated is obtained.

[0053] The advantages of the present invention are:

[0054] (1) The present invention derives a correlation equation between the smear formation principle and transfer rules and the original image. The camera is pre-calibrated to obtain the specific parameters of the correlation equation, thereby deriving and eliminating the smear based on the original image. This method only requires a single-frame image and can process multiple frames frame by frame. The elimination process is fast and simple, and after a single calibration, it can be used for shooting at various exposure times.

[0055] (2) After calibration in advance, the smear of the real images taken at different exposure times can be eliminated.

[0056] (3) The data used for elimination only require the original image and exposure time in addition to the parameters obtained by calibration. The elimination process is highly automated and does not require target positioning and manual selection of dark pixels. It can be implemented using a simple programming language.

[0057] (4) Only a single frame image is needed to eliminate noise, and no joint processing of multiple frame images is performed. The processing takes up less resources and the steps are simple and quick. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic structural diagram of a typical interline transfer CCD to which the present invention is applicable is shown;

[0059] Figure 2 FIG1 shows a schematic diagram of charge transfer in the first frame of a typical inter-line transfer CCD to which the present invention is applicable;

[0060] Figure 3 Schematic diagram showing charge transfer in the second frame and subsequent frames of a typical interline transfer CCD to which the present invention is applicable;

[0061] Figure 4 The overall flow chart of calibration and elimination of smear of an inter-line transfer CCD camera according to an embodiment of the present invention is shown;

[0062] Figure 5 A flowchart illustrating the derivation of a correlation equation between the smear image and the original image of an inter-line transfer CCD camera according to an embodiment of the present invention is shown;

[0063] Figure 6 A flow chart of calibration of smear of an inter-line transfer CCD camera according to an embodiment of the present invention is shown;

[0064] Figure 7 A flow chart of eliminating smear of an inter-line transfer CCD camera according to an embodiment of the present invention is shown.

[0065] In the picture:

[0066] 1. Pixel; 2. Photodiode; 3. Vertical CCD; 4. Horizontal CCD; 5. Output amplifier. DETAILED DESCRIPTION

[0067] like Figure 4 As shown, a method for calibrating and eliminating the smear of an inter-line transfer CCD camera includes the following steps:

[0068] S1: Based on the formation principle of smear and original image, the relationship between the smear s(i,j) generated by the charge transfer of each pixel and the effective image g(i,j) after eliminating the smear is derived. The relationship between the smear s(i,j) and the original image m(i,j) is derived from the transfer rule of the smear, and the smear uniformly affected by the pixels in the jth column is derived. and the accumulated value of the jth column of the original image related relationships; such as Figure 5 The specific steps are as follows:

[0069] S11. The specific steps for obtaining the relationship between s(i,j) and g(i,j) are as follows:

[0070] The expression of the smear s(i,j) is

[0071] s(i,j)=η*k*t1*r(i,j) (1)

[0072] The expression of the effective image g(i,j) is

[0073] g(i,j)=k*t0*r(i,j)+b (2)

[0074] According to (1) and (2), the relationship between s(i,j) and g(i,j) is

[0075]

[0076] Where k is the camera's photoelectric conversion parameter; t0 is the exposure time; t1 is the charge transfer time per pixel; r(i,j) is the luminous flux per pixel; b is the image background noise caused by dark current; η is the ratio parameter between g(i,j) and s(i,j);

[0077] S12. The specific steps for obtaining the relationship between s(i,j) and m(i,j) are as follows:

[0078] Assume that the charge generated by the light-sensitive pixels in the jth column, i.e., the effective image of the jth column, is [g1, g2, g3, ..., g N-2 ,g N-1 ,g N ], where N is the number of pixels in a column; the corresponding smear noise charge generated by the charge transfer of each pixel is [s1,s2,s3,...,s N-2 ,s N-1 ,s N ].Depend on Figure 2 It can be seen that in the first frame of charge transfer, the output charge of this column is

[0079] When the charge is transferred in the vertical CCD, noise charge is continuously introduced due to the incomplete light shielding; however, when it is transferred to the horizontal CCD, no new noise charge is introduced. Figure 3 It can be seen that after the output charge of N pixels is transferred in the first frame, there is still residual noise charge in the vertical CCD that has not been transferred and output and affects the next frame. Therefore, starting from the second frame, the output charge of this column is

[0080] Since the same amount of residual charge in each subsequent frame affects the next frame, this formula can represent the output charge of this column in each subsequent frame. Since image acquisition generally starts several frames after the CCD starts to transfer charge, it can be considered that the smear transfer law in the acquired image follows Figure 3 , that is, the smear effect on the original image is manifested as: on top of the original photosensitive charge of each pixel, the sum of the smears suffered by each pixel in this column during the transfer is added, and we can get (4):

[0081]

[0082] It should be noted that: Figure 2 , as shown in 3, s(i,j) is the smear noise charge introduced when the original charge of pixel (i,j) is transferred one pixel downward, and does not represent the smear of pixel (i,j) on the final image. From (4), we can see that the smear of pixel (i,j) on the image is The smear that appears in the following text, unless otherwise specified, occurs during single pixel charge transfer.

[0083] S13, derive the uniform smear of the pixels in the jth column and the accumulated value of the jth column of the original image The specific steps are as follows:

[0084] S131. Combining (3) and (4), we can get (5):

[0085]

[0086] S132. Performing a single-column accumulation operation on both sides of equation (5) yields (6):

[0087]

[0088] S133. Simplify (6) to (7):

[0089]

[0090] S134, from (7) we can know and There is a linear correlation. Let the slope of this linear relationship be K. Since K is related to t0, K is further expressed as K(t0). Let the x-intercept be B, which is independent of t0, then we have the following formula:

[0091]

[0092] B=-N*b (9)

[0093]

[0094] Step S2: Use the camera to shoot, obtain multiple scatter plots at different t0, and fit them to get the smear and the accumulated value of the jth column of the original image The specific relationship, such as Figure 6 The specific steps are as follows:

[0095] S21. Use the camera to shoot, exposure time t0, obtain a frame of original image m(i,j) in the calibration, and calculate the cumulative value of each column of the original image

[0096] S22. Measure the total amount of smear that each pixel in the image experiences during transfer. The specific steps are as follows:

[0097] S221: Manually select dark reference rows in the dark area of ​​m(i, j), which are generally 10 to 100 rows located at the upper and lower edges of the image. In this solution, since the camera calibration processing object is a single image, no complex positioning processing is required, so manual selection of dark reference rows is chosen to obtain accurate smear.

[0098] S222: Averaging the dark reference lines to obtain image smear

[0099] S23: Take photos at multiple exposure times and repeat S21 to S22 to obtain the corresponding values ​​of different t0. and

[0100] S24: is the independent variable, As the dependent variable, draw multiple scatter plots at different t0;

[0101] S25: Perform linear fitting on multiple scatter plots at different t0 to obtain the slope K(t0) and the x-intercept B. During the fitting, since the x-intercept B does not change with t0, multiple plots should have a unified x-intercept B. Therefore, a global fitting is required for the multiple plots to share the x-intercept B, and K(t0) and unique B corresponding to different t0 are obtained. In this solution, the fitting method is the least squares method.

[0102] S26: Take t0 as the independent variable and K(t0) as the dependent variable, make a scatter plot, use formula (8) as the relationship to perform inverse proportional fitting, obtain the parameter η*t1, and determine and The specific relationship of When it is at a low level, it means that the pixel (i, j) is located in a low light area. The values ​​are also extremely low, indicating significant errors in the smear measured during calibration. The initial data may not be linearly correlated. Due to experimental error, the linear correlation of the final data may be poor. Generally, when the goodness-of-fit of one or more scatter plots and the overall goodness-of-fit are less than 0.9 at different t0s, recalibration is necessary. This is considered effective and the smear reduction effect is reliable.

[0103] Step S3: Eliminate the trailing image during camera shooting. Figure 7 The specific steps are as follows:

[0104] S31: Use the camera to shoot, knowing the exposure time T0 and the original image M(i, j) corresponding to the actual shot, calculate according to step S2

[0105] S32: Substitute T0 into (8) with known η*t1 to obtain the corresponding K(T0);

[0106] S33: Substitute the calculated K(T0) and the known B in S25 into (10) to obtain the smear and The specific relationship between Find the value of each column

[0107] S34: Subtract the number of columns from the M(i,j) formula corresponding to each pixel. Then the image G(i,j) with the smear eliminated is obtained.

[0108] Regarding the derivation of the correlation between image smear and the original image in this embodiment, it should be noted that the premise of this derivation is that the transfer rule of the smear is as follows: Figure 2 、 3 As shown, the following requirements are met: 1. The smear is mainly caused by incomplete shading of the vertical CCD; 2. The objects photographed during calibration and actual shooting do not change significantly within the single exposure time and single frame transfer time.

[0109] Regarding the camera calibration of the correlation parameters in this embodiment, it should be noted that: 1. Even for cameras of the same model, there may be differences in parameters such as dark current noise, photoelectric conversion coefficient, pixel transfer time, etc., so each camera needs to be calibrated separately; 2. Calibration should be performed after the camera is assembled and shipped from the factory. The calibration result can appropriately refer to the relevant factory parameters of the camera based on the derived relationship; 3. The result of a single calibration is applicable to actual measurements with the same light source band, different shooting objects, and different exposure times. That is, if the light source band changes during the actual measurement, the calibration needs to be repeated. If the shooting object and exposure time change during the actual measurement, the calibration does not need to be repeated.

[0110] In this embodiment, the ghosting is eliminated during actual camera shooting. It should be noted that the elimination step can be implemented using a simple programming language and can be inserted into the image visualization algorithm. The calculation time of the elimination step is affected by conditions such as image size and hardware and is generally on the order of microseconds to milliseconds, and usually does not affect the real-time visualization and other real-time processing of the image.

[0111] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for calibrating and eliminating smear of an inter-line transfer CCD camera, characterized in that: The following steps are involved: S1: Based on the formation principle of smear and original image, the relationship between the smear s(i,j) generated by the charge transfer of each pixel and the effective image g(i,j) after eliminating the smear is derived. The relationship between the smear s(i,j) and the original image m(i,j) at the time of calibration is derived based on the smear transfer rule. Finally, the uniform smear of the pixels in the jth column is derived. and the accumulated value of the jth column of the original image the relevant relationship; S2: Calibrate the camera, use the camera to shoot, obtain multiple scatter plots at different exposure times t0, and fit them to get the smear and the accumulated value of the jth column of the original image The specific relationship of S3: Eliminate smear when shooting with the camera; The specific steps of step S1 are: S11. The specific steps for obtaining the relationship between s(i,j) and g(i,j) are as follows: The expression of the smear s(i,j) is s(i,j)=η*k*t1*r(i,j) (1) The expression of the effective image g(i,j) is g(i,j)=k*t0*r(i,j)+b (2) According to (1) and (2), the relationship between s(i,j) and g(i,j) is Where k is the camera's photoelectric conversion parameter; t0 is the exposure time; t1 is the charge transfer time per pixel; r(i,j) is the luminous flux per pixel; b is the image background noise caused mainly by dark current; η is the ratio parameter between g(i,j) and s(i,j); S12. The specific steps for obtaining the relationship between s(i,j) and m(i,j) are as follows: Assume that the charge generated by the light-sensitive pixels in the jth column, i.e., the effective image of the jth column, is [g1, g2, g3, ..., g N-2 ,g N-1 ,g N ], where N is the number of pixels in a column; the corresponding smear noise charge generated by the charge transfer of each pixel is [s1,s2,s3,...,s N-2 ,s N-1 ,s N ]; In the first frame of charge transfer, the output charge of this column is Starting from the second frame, the output charge of this column is The smear effect on the original image is as follows: the sum of the smears that each pixel in this column experiences during the transfer is added to the original photosensitive charge of each pixel, and we can get (4): S13, derive the uniform smear of the pixels in the jth column and the accumulated value of the jth column of the original image the relevant relationship; The specific steps of step S13 are as follows: S131. Combining (3) and (4), we can get (5): S132. Performing a single-column accumulation operation on both sides of equation (5) yields (6): S133. Simplify (6) to (7): S134. Let the slope of the linear relationship of formula (7) be K, and further express K as K(t0); and let the x-axis intercept be B, which is independent of t0, then the following formula is obtained: B=-N*b (9) Step S3 is specifically as follows: S31: Use the camera to shoot, knowing the exposure time T0 and the original image M(i, j) corresponding to the actual shot, calculate according to step S2 S32: Substitute T0 into (8) with known η*t1 to obtain the corresponding K(T0); S33: Substitute the calculated K(T0) and the known B into (10) to obtain the smear and The specific relationship between Find the value of each column S34: Subtract the number of columns from the M(i,j) formula corresponding to each pixel. Then the image G(i,j) with the smear eliminated is obtained.

2. The method for calibrating and eliminating streaking of an inter-line transfer CCD camera according to claim 1, characterized in that: Step S2 is specifically as follows: S21. Use the camera to shoot, obtain a frame of original image m(i,j) in the calibration, and calculate the cumulative value of each column of the original image S22. Measure the total amount of smear that each pixel in the image experiences during transfer. S23: Take photos at multiple exposure times and repeat S21 to S22 to obtain the corresponding values ​​of different t0. and S24: is the independent variable, As the dependent variable, draw multiple scatter plots at different t0; S25: Perform linear fitting on multiple scatter plots at different t0 to obtain the slope K(t0) and the x-axis intercept B; S26: Take t0 as the independent variable and K(t0) as the dependent variable, make a scatter plot, use formula (8) as the relationship to perform inverse proportional fitting, obtain the parameter η*t1, and determine and The specific relationship of .

3. The method for calibrating and eliminating streaking of an inter-line transfer CCD camera according to claim 2, wherein: Step S22 is specifically as follows: S221: Manually select the dark reference row in the dark area of ​​m(i,j); S222: Averaging the dark reference rows to obtain image smear 4. The method for calibrating and eliminating streaking of an inter-line transfer CCD camera according to claim 3, wherein: In step S221 , the dark reference lines are located at lines 10 to 100 at the upper and lower edges of the image.

5. The method for calibrating and eliminating smear of an inter-line transfer CCD camera according to claim 2, characterized in that: The fitting method in step S25 is the least squares method.

Images