A real-time method for removing vertical stripes from infrared images using improved local moment matching

Abstract

The present invention discloses a method for removing vertical stripes from infrared images in real time by utilizing improved local moment matching. The method is applicable to gun sights and artillery sights equipped with uncooled infrared detectors. In terms of hardware, the uncooled infrared detector has an infrared core hardware with an image acquisition frame frequency of 50 Hz. Based on the video frequency of 25 Hz recognizable by the human eye, the frame rate of the observed video is determined to be 50 Hz by combining the advantages of time-domain algorithm processing of the infrared image without affecting the final output observation video frame rate. The infrared core hardware has a baffle correction function, which can correct and compensate for the non-uniformity of the infrared image. The infrared focal plane detector uses a shared amplifier in the same column. Compared with the prior art, the present invention utilizes this method on the core component of the uncooled infrared detector to filter out the vertical stripe noise of the infrared detector in real time, thereby improving the infrared image detection capability.

Description

Technical Field

[0001] The present invention relates to the field of infrared imaging technology, and in particular to a method for removing vertical stripes of infrared images in real time by utilizing improved local moment matching.

[0002] This algorithm is suitable for gun sights and artillery sights with uncooled infrared detectors. By using the method of local matching to remove vertical stripes in infrared images, it can automatically remove the striped non-uniform noise caused by the limitations of existing manufacturing process levels and materials, and the inconsistency of the response characteristics of each detector unit in the infrared focal plane array, thereby improving infrared detection capabilities. Background Art

[0003] Uncooled infrared detectors have been widely used in sights and other observation instruments. However, due to the limitations of existing manufacturing process levels and materials, the infrared focal plane adopts an integral form. The infrared focal plane detector contains multiple amplifiers, but usually in order to save costs, the outputs of the same row or column share one amplifier (the detector used in the hardware of the present invention shares one amplifier in the same column direction), that is, the integral current of each column of pixels corresponds to the same bias voltage VFID. Because the bias voltage has noise, the gate voltage obtained by the FET gate during the integration of the front and rear columns is not the same. This causes the integral current ip flowing through each column of pixels to be different even if the same infrared radiation is obtained. This is the root cause of the stripe noise in the infrared image output by this type of infrared detector. Different columns of detection photosensors will produce different output signals for the same infrared radiation, which results in a stripe-like non-uniform noise in the infrared image. The vertical stripes in the image will seriously affect the use of the product and are not conducive to the observer's recognition of the target. Therefore, it is necessary to remove the vertical stripes from the infrared image in real time. The conventional method for removing vertical stripes is mainly based on the moment matching method. However, the traditional moment matching method will introduce a "banding effect" as follows Figure 1 As shown, in order to solve the "banding effect", the present invention proposes an improved local moment matching method to remove vertical stripes and greatly eliminate the "banding effect" at the same time. Summary of the Invention

[0004] In view of the defect of existing traditional moment matching methods in dealing with vertical stripes in infrared images, the purpose of the present invention is to provide a method for removing vertical stripes in infrared images in real time by using improved local moment matching. On the core component of an uncooled infrared detector, this method is used to filter out the vertical stripe noise of the infrared detector in real time, thereby improving the infrared image detection capability.

[0005] To achieve the above objectives, the technical solution of the present invention is: a method for removing vertical streaks in infrared images in real time using improved local moment matching, which is applicable to gun sights and artillery sights with uncooled infrared detectors. In terms of hardware:

[0006] The uncooled infrared detector has an infrared core hardware with an image acquisition frame frequency of 50 Hz. Based on the video frequency of 25 Hz that can be recognized by the human eye, and without affecting the final output observation video frame rate, the infrared image is processed in the time domain algorithm. The frame rate is determined to be 50 Hz.

[0007] The infrared core hardware has a baffle correction function, which can correct and compensate for the non-uniformity of infrared images; the infrared focal plane detectors use a shared amplifier in the same column;

[0008] Using the original data collected by 14-bit AD as the original data for image processing can effectively retain the image details and facilitate the analysis of infrared image original data;

[0009] In terms of software, the traditional moment matching method is improved and the local window moment matching method is used to filter out vertical stripe noise. This method can ensure the effectiveness of the moment matching method in removing vertical stripe noise in images, cope with complex environment images, and eliminate the "banding effect";

[0010] Combined with the actual application scenario, the column mean distribution between the columns adjacent to the local window is approximately equal. In particular, the smaller the local window, the closer the column mean is to uniformity. Therefore, the present invention uses two adjacent columns as reference columns, that is, the first two columns of the current column. This method effectively avoids the phenomenon of grayscale distortion in the image caused by improper selection of reference columns. The specific analysis of image filtering is as follows:

[0011] Let the image window size be R, then: n∈[1,row / R]. After the image completes the basic non-uniformity correction, traverse the image data X(i,j) twice, and calculate the final output image grayscale value Y(i,j) according to the following steps, where i,j are row and column indices, δ(n,j),u(n,j) are the local column mean square error and expectation of the jth column, row is the total number of rows of the infrared image, and n is the nth window that divides the image into row / R local panes. The specific steps are:

[0012] 1) Determine the value of n by traversing the i index of the data X(i, j). The value of n is determined according to the following formula:

[0013] n=int(i / 16)+1 (4)

[0014] 2) Calculate the local mean and variance of the nth window as follows:

[0015]

[0016]

[0017] 3) Traverse the data X(i, j) again and calculate the final output according to the following process:

[0018] When j is greater than 2, the calculation formula is:

[0019] K=δ (n,j-1) δ (n,j-2) / δ (n,j) 2 (7)

[0020]

[0021] The "striping effect" caused by selecting a fixed reference column is eliminated by using the moment matching method of the local window, and vertical stripe noise is effectively eliminated at the same time;

[0022] When j is less than 2, the calculation formula is:

[0023] K=δ (n,j+1) δ (n,j+2) / δ (n,j) 2 (9)

[0024]

[0025] According to the processing methods of the above two formulas (9) and (10), the first two columns at the edge of the image can be effectively processed. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The specific embodiments of the present invention are further described in detail below with reference to the accompanying drawings.

[0027] Figure 1 This is the stripe effect caused by the traditional moment matching method before and after the image is processed by the present invention (the left side is the image before processing, and the right side is the image after processing).

[0028] Figure 2 This is the comparison of images before and after processing by the improved local moment matching method of the present invention (the left side is the image before processing, and the right side is the image after processing). DETAILED DESCRIPTION

[0029] Attachment Figure 1 In the conventional method, the moment matching method processes vertical stripes according to the following process:

[0030] Since vertical stripe noise is mainly caused by the inconsistent response functions of each detection unit within the spectral response area, it is generally distributed as vertical stripes on the original image without periodicity. When the surface signal is relatively weak, this noise will be more obvious. Considering that noise is generally superimposed on the signal, the noise is separated based on the statistical results of the image at different scales.

[0031] Assuming that the scene detected by each detector is in an ideal situation with the same balanced radiation distribution, the change of the data recorded on the detector is linearly related to the gain coefficient and bias coefficient of the radiation correction and is shift-invariant. If Ci is the i-th detection unit, the spectral response function of Ci can be expressed as:

[0032] Y i =G i X+O i +ε i (X) (1)

[0033] In formula (1), Yi represents the output value of Ci, that is, the grayscale value of the image; X represents the radiation value received by the system, that is, the real scene value; Gi represents the gain coefficient; Oi represents the bias coefficient; εi represents random noise. When the signal-to-noise ratio is high, the influence of εi can be ignored, and formula (1) can be written as:

[0034] Y i =G i X+O i (2)

[0035] It can be seen from formula (2) that when the radiation intensity X has the same value, if the gain coefficient Gi and the bias coefficient Oi take different values, the obtained grayscale value Yi will also be different, which will cause the appearance of vertical stripes. Therefore, if the grayscale value Yi is normalized to the same value, the vertical stripe noise can be effectively eliminated.

[0036] Under the assumption that the scene is uniform, the mean and variance of the incident radiation intensity of each column are approximately equal. The moment matching method is to select a column as a reference, and then adjust the mean and variance of other detector columns to the radiance of the reference column according to formula (2).

[0037]

[0038] In formula (3), Xi and Yi represent the grayscale values ​​of the pixels in the i-th column of the image before and after correction, respectively; μr and σr represent the mean and standard deviation of the reference row, respectively; μi and σi represent the mean and standard deviation of the i-th row, respectively.

[0039] According to the above algorithm, the moment matching method is relatively effective in removing image stripe noise, but it is based on the condition that the image scene is uniformly distributed. In the case of uneven grayscale distribution caused by complex scenes or rich image content, the moment matching method usually produces a "banding effect". The fundamental reason for this phenomenon is that after the moment matching method, all the column means of the image are approximately equal, which changes the distribution of the image column means and makes it approximately distributed into a straight line. Since it is generally impossible to have a uniform distribution in the image, it is difficult to achieve such a straight grayscale distribution. Therefore, if the reference column is not selected properly, the grayscale information of the original image will change, resulting in grayscale distortion, and then a "banding effect". As shown in the attached figure Figure 1 shown.

[0040] Attachment Figure 2 This is an embodiment of the present invention, which discloses a method for removing vertical stripes from infrared images in real time using improved local moment matching.

[0041] Let the image window size be R = 16, then: n∈[1,row / 16]. After the image completes the basic non-uniformity correction, traverse the image data X(i,j) twice, and calculate the final output image grayscale value Y(i,j) according to the following steps, where i,j are row and column indices, δ(n,j),u(n,j) are the local column mean square error and expectation of the jth column, row is the total number of rows of the infrared image, and n is the nth window that divides the image into row / R local panes. The specific steps are:

[0042] 1) Determine the value of n by traversing the i index of the data X(i, j). The value of n is determined according to the following formula:

[0043] n=int(i / 16)+1 (4)

[0044] 2) Calculate the local mean and variance of the nth window as follows:

[0045]

[0046]

[0047] 3) Traverse the data X(i, j) again and calculate the final output according to the following process:

[0048] When j is greater than 2, the calculation formula is:

[0049] K=δ (n,j-1) δ (n,j-2) / δ (n,j) 2 (7)

[0050]

[0051] The "striping effect" caused by selecting a fixed reference column is eliminated by using the moment matching method of the local window, and vertical stripe noise is effectively eliminated at the same time;

[0052] When j is less than 2, the calculation formula is:

[0053] K=δ (n,j+1) δ (n,j+2) / δ (n,j) 2 (9)

[0054]

[0055] According to the above two formulas (9) and (10), the first two columns at the edge of the image can be effectively processed. Figure 2 The "banding effect" in the image has been basically eliminated.

[0056] The above description is only a preferred embodiment of the present invention. The above specific embodiments are not limitations of the present invention. Any modifications, changes or equivalent substitutions made by ordinary technicians in this field based on the above description are within the scope of protection of the present invention.

Claims

1. A method for removing vertical streaks from infrared images in real time using improved local moment matching, suitable for use in gun sights and artillery sights with uncooled infrared detectors, characterized by: Let the image window size be R = 16, then: n∈[1, row / R]. After the image completes the basic non-uniformity correction, traverse the image data X(i, j) twice, and calculate the final output image grayscale value Y(i, j) according to the following steps, where i and j are row and column indices, δ(n, j), u(n, j) are the local column mean square error and expectation of the jth column, row is the total number of rows of the infrared image, and n is the nth window that divides the image into row / R local panes. The specific steps are: 1) Determine the value of n by traversing the i index of the data X(i, j). The value of n is determined according to the following formula: n=int(i / 16)+1 (1) 2) Calculate the local mean and variance of the nth window as follows: 3) Traverse the data X(i, j) again and calculate the final output according to the following process: When j is greater than 2, the calculation formula is: K=δ (n,j-1) d (n,j-2) d (n,j) 2 (4) The "striping effect" caused by selecting a fixed reference column is eliminated by using the moment matching method of the local window, and vertical stripe noise is effectively eliminated. When j is less than 2, the calculation formula is: K=δ (n,j+1) d (n,j+2) d (n,j) 2 (6) According to the processing methods of the above two formulas (6) and (7), the first two columns at the edge of the image can be effectively processed.

Images