Infrared image non-uniformity correction method and system based on FPGA

Abstract

The present invention provides an infrared image non-uniformity correction method and system based on FPGA, comprising the following steps: step S1: obtaining the cumulative histogram value of the original image; step S2: calculating the new value according to the cumulative histogram value; step S3: mapping the new value in combination with the original image to obtain the final image. Based on the special structure of FPGA and the traditional median histogram algorithm, the present invention proposes an infrared image non-uniformity correction method based on FPGA, which can effectively remove the stripe noise on the original infrared image and is convenient to implement on FPGA.

Description

Technical Field

[0001] The present invention relates to the technical field of infrared image correction, and in particular to an FPGA-based infrared image non-uniformity correction method and system. Background Art

[0002] With the development of focal plane arrays and the continuous maturity of image processing technology, the application of infrared thermal imaging systems is becoming more and more extensive. However, due to the limitations of the manufacturing process in infrared imaging systems, the photoelectric responses of the detection units on the focal plane array to the same irradiation are not completely consistent, resulting in large noise, low contrast, poor visual effects, and poor ability to distinguish image details, which seriously affects the imaging quality of the infrared system. Stripes are a relatively obvious inherent noise in the original infrared image.

[0003] The Chinese invention patent document with publication number CN110211056A discloses an adaptive infrared image stripe removal algorithm based on a local median histogram. By calculating the scene complexity in the sliding window, the window with the smallest scene change between each column and the adjacent column in the original infrared image is found. The cumulative histogram of all columns in the window is calculated to obtain the median histogram of the center column of the window. The corrected grayscale value of each pixel in the center column of the window is calculated based on the median histogram. The stripe value of the column can be obtained by subtracting the corrected grayscale value mean from the original grayscale value mean. Subtracting the stripe value from the entire column is the final output grayscale value of the column.

[0004] With respect to the above-mentioned related technologies, the inventor believes that the traditional median histogram stripe removal method has a complicated calculation process and is not convenient to implement on an FPGA. Summary of the invention

[0005] In view of the defects in the prior art, the object of the present invention is to provide an infrared image non-uniformity correction method and system based on FPGA.

[0006] According to the present invention, a method for correcting infrared image non-uniformity based on FPGA includes the following steps:

[0007] Step S1: Obtain the cumulative histogram value of the original image;

[0008] Step S2: Calculate a new value based on the cumulative histogram value;

[0009] Step S3: Mapping is performed according to the new value combined with the original image to obtain a final image.

[0010] Preferably, step S1 comprises the following steps:

[0011] Step S1.1: Obtain the histogram value of each row or column of the original image;

[0012] Step S1.2: Calculate the cumulative histogram value of each row of the original image according to the histogram value of each row of the original image in step S1.1; or,

[0013] The cumulative histogram value of each column of the original image is calculated according to the histogram value of each column of the original image in step S1.1.

[0014] Preferably, step S2 comprises the following steps:

[0015] Step S2.1: Determine the number of rows or columns of the cumulative histogram values ​​involved in the calculation;

[0016] Step S2.2: Obtain a new value by determining the cumulative histogram value of the number of rows or columns.

[0017] Preferably, in step S2.2, the cumulative histogram values ​​of the determined number of rows or columns are subjected to an inverse transformation of the cumulative histogram values, and the inverse transformed cumulative histogram values ​​are averaged using the quartile method to obtain new values.

[0018] Preferably, in step S2.2, the inverse transformed cumulative histogram values ​​are sorted to obtain sorted data;

[0019] The sorted data is divided equally by Q1, Q2 and Q3, where each part contains the sorted data divided equally; the calculation formula for obtaining Q1, Q2 and Q3 is:

[0020] Q3=Q2-1+Q1;

[0021] Among them, n represents the number of sorted data; Q1 represents the corresponding position of the quartile P0.25; Q2 represents the corresponding position of the quartile P0.5; Q3 represents the corresponding position of the quartile P0.75; P0.25 represents the data at the one-fourth position of the sorted data; P0.5 represents the data at the second-fourth position of the sorted data; P0.75 represents the data at the three-fourth position of the sorted data.

[0022] Preferably, in step S2.2,

[0023] s = k*(P0.75-P0.25);

[0024] Where s represents the standard deviation of the sorted data; k represents the reciprocal of the standard normal interquartile range;

[0025] The minimum estimated value of the sorted data is min=P0.75+1.5*s, the maximum estimated value of the sorted data is max=P0.25-1.5*s, and the new value is equal to (max+min) / 2.

[0026] Preferably, in step S3, grayscale mapping is performed by combining the new value, the original cumulative histogram and the original image to obtain a final image;

[0027] The mapping relationship is:

[0028] final(i,j)=result(Chist(image(i,j)+1,j),j);

[0029] Among them, (i, j) represents the i-th row and j-th column, image represents the original image, Chist represents the original cumulative histogram image; result represents the results of the first four steps, and final represents the final image.

[0030] According to the present invention, an infrared image non-uniformity correction system based on FPGA includes the following modules:

[0031] Module M1: Get the cumulative histogram value of the original image;

[0032] Module M2: Calculate a new value based on the cumulative histogram value;

[0033] Module M3: Mapping is performed according to the new value combined with the original image to obtain a final image.

[0034] Preferably, the module M1 includes the following modules:

[0035] Module M1.1: Get the histogram value of each row or column of the original image;

[0036] Module M1.2: Calculate the cumulative histogram value of each row of the original image according to the histogram value of each row of the original image in the module M1.1; or,

[0037] The cumulative histogram value of each column of the original image is calculated according to the histogram value of each column of the original image of the module M1.1.

[0038] Preferably, the module M2 includes the following modules:

[0039] Module M2.1: Determine the number of rows or columns of the cumulative histogram values ​​involved in the calculation;

[0040] Module M2.2: Obtain new values ​​by determining the cumulative histogram value of the number of rows or columns.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. Based on the special structure of FPGA and the traditional median histogram algorithm, the present invention proposes an infrared image non-uniformity correction algorithm based on FPGA, which can effectively remove the stripe noise on the original infrared image and is convenient to implement on FPGA;

[0043] 2. The present invention takes improving the quality of infrared images as a starting point, can effectively remove the stripe noise on the original infrared images, and at the same time can improve the contrast, enhance the infrared images, and achieve good results. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0045] Figure 1 It is a flow chart of the present invention;

[0046] Figure 2 It is a mapping relationship diagram of the present invention;

[0047] Figure 3 It is a comparison chart of the original image, the original algorithm effect image and the algorithm effect image of the present invention. DETAILED DESCRIPTION

[0048] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0049] The embodiment of the present invention discloses a method for correcting infrared image non-uniformity based on FPGA. Figure 1 and Figure 2 As shown, the following steps are included: Step S1: Obtain the cumulative histogram value of the original image, and obtain the original cumulative histogram value. Step S1 includes the following steps: Step S1.1: Obtain the histogram value of each row of the original image or each column of the original image. If the original image is vertical stripes, column processing is used, and if the original image is horizontal stripes, row processing is used. Obtain the histogram value of each row or column (stripe direction). The full name of FPGA is Field Programmable Gate Array, and its Chinese translation is field editable logic gate array.

[0050] Step S1.2: Calculate the cumulative histogram value of each row of the original image according to the histogram value of each row of the original image in step S1.1; or, calculate the cumulative histogram value of each column of the original image according to the histogram value of each column of the original image in step S1.1. Calculate the cumulative histogram value of each row or column (striping direction).

[0051] Step S2: Calculate a new value based on the cumulative histogram value. Step S2 includes the following steps: Step S2.1: Group the cumulative histogram values ​​and determine the number of rows or columns of the cumulative histogram values ​​involved in the calculation. Determine the number of rows or columns involved in the calculation. Here, 5 columns are taken as an example, that is, the target column (determine the number of columns of the cumulative histogram values ​​involved in the calculation) has a window size of 5, and the center column is the 3rd column.

[0052] Step S2.2: Obtain a new value by determining the cumulative histogram value of the number of rows or columns. Calculate the average of the histogram by determining the cumulative histogram value of the number of rows or columns.

[0053] If the original median histogram algorithm principle is used:

[0054]

[0055] Among them, H j Cumulative histogram for a column, Represents the inverse process of the histogram, such as: H j (222) = 10, which means that the number of points with gray value 222 is 10, then The grayscale value corresponding to 10 points is 222; where Φ(j) is the weight for weighted averaging. The weight in the original median histogram algorithm uses the Gaussian weight formula:

[0056]

[0057] Among them, sqrt represents the square root, exp represents the power of e, e represents the base of the natural logarithm, and d represents the distance between the current column and the center column of the window. For example, the distance between the first column and the third column of the center column is 2. Sigma is the smoothing coefficient, which needs to be determined manually according to the image noise situation. However, the Gaussian function calculation process is more complicated to implement on FPGA, so here we use another method of averaging, the quartile method.

[0058] Perform an inverse transformation on the cumulative histogram values ​​of a certain number of rows or columns, and use the quartile method to average the inverse transformed cumulative histogram values ​​to obtain a new value. Sort the inverse transformed cumulative histogram values ​​to obtain sorted data. Perform an inverse histogram transformation,

[0059]

[0060] in, Indicates the average value of the cumulative histogram of the inverse transformation; aver indicates the average value; H j is the cumulative histogram value of the jth column, Represents the inverse process of the cumulative histogram value of the jth column. (j can be replaced by i to represent the i-th row).

[0061] If there is Updated to That is to say, the data in the middle of a set of data should be replaced by the average value. That is, the value in the middle of this set of data is replaced by the average value, and the value in the middle of this set of data is corrected by the average value of a set of data. There are many ways to calculate the average. The original algorithm uses weighted average, which is complex and not suitable for FPGA implementation because it contains exponential operations. Here, the quartering method is used to calculate the average, which is convenient for FPGA implementation.

[0062] The basic principle of the quartile method: sort a set of data from small to large, and divide all the data into 4 parts through 3 position points (Q1, Q2, Q3), each of which contains 25% of the data. These 3 position points (Q1, Q2, Q3) correspond to the positions of the quartiles (P0.25, P0.5, P0.75). There are many ways to calculate Q1, Q2, and Q3. The following is used here, where n is the number of data (n is different for odd and even numbers, because the image window size is usually an odd number, so only the odd number calculation is explained here).

[0063] The sorted data is divided equally by Q1, Q2 and Q3, where each part contains the sorted data divided equally; the calculation formula for obtaining Q1, Q2 and Q3 is:

[0064] Q3=Q2-1+Q1.

[0065] Where n represents the number of sorted data; Q1 represents the position corresponding to the quartile P0.25; Q2 represents the position corresponding to the quartile P0.5; Q3 represents the position corresponding to the quartile P0.75; P0.25 represents the data at the one-quarter position of the sorted data; P0.5 represents the data at the one-second position of the sorted data; P0.75 represents the data at the three-fourth position of the sorted data. Q represents the position in a set of data, P represents the data at the Q position, P0.25 is at the Q1 position, and the Q1 position is the one-quarter position of this set of data.

[0066] For example: Suppose a set of data is [1 4 1 5 6 2 7], arranged from small to large as [1 1 2 4 5 6 7], the number of data is 7, according to the above formula, Q1=2.5, Q2=4, Q3=5.5, and the corresponding P0.25=(1+2) / 2=1.5 (the second number arranged from small to large is 1, the third number is 2, so the 2.5th number is the average of the two), P0.5=4, P0.75=(5+6) / 2=5.5.

[0067] If n = 5, There are 5 numbers in total. Use the quartering method to find the average of these 5 numbers.

[0068] Q3=Q2-1+Q1=4.

[0069] Then P0.25 corresponds to the number at position Q1, that is, P0.25 is the second number of the data. Similarly:

[0070] Next, we need to find the average value of the data set. First, we need to find the standard deviation of the data set, s = k*(P0.75-P0.25), where k is obtained from the standard normal distribution. The interquartile range of the standard normal distribution is -0.6745 to 0.6745, so the width is 0.6745*2 = 1.3490, k = 1 / 1.3490 = 0.7513, so the interquartile range of a certain data set (the interquartile range of a certain data set = P0.75-P0.25) multiplied by 0.7513 is equivalent to the standard deviation of the data set. The minimum estimated value of a certain data set is min = P0.75 + 1.5*s, and the maximum estimated value is max = P0.25-1.5*s. Therefore, the average value of a certain data set can be approximately replaced by (max+min) / 2.

[0071] s=k*(P0.75-P0.25), where s represents the standard deviation of the sorted data; k represents the inverse of the standard normal interquartile range. The minimum estimated value of the sorted data min=P0.75+1.5*s, and the maximum estimated value of the sorted data max=P0.25-1.5*s; the new value is equal to (max+min) / 2. The new value is the average processed value.

[0072] Step S3: Mapping is performed based on the new value and the original image to obtain the final image. Grayscale mapping is performed on the new value, the original cumulative histogram (initial cumulative histogram) and the original image to obtain the final image. Grayscale mapping is performed on the previous result, the initial cumulative histogram and the original image to obtain the final result.

[0073] The mapping relationship is:

[0074] final(i,j)=result(Chist(image(i,j)+1,j),j).

[0075] Among them, (i, j) represents the i-th row and j-th column, image represents the original image, Chist represents the original cumulative histogram image; result represents the results of the first four steps, and final represents the final image.

[0076] like Figure 3As shown in the figure, it is a comparison of the original image, the original algorithm effect image and the algorithm effect image of the present invention. The top is the original image, the middle is the original algorithm effect image; the bottom is the algorithm effect image of the present invention. It can be seen from the result that the effect of the improved algorithm is slightly lower than that of the original algorithm, but the process is much simpler.

[0077] The purpose of the present invention is to optimize the traditional median histogram stripe removal algorithm process, so that it is easy to implement in FPGA and does not affect the original effect. The cumulative histogram value of a single row (or single column) image in the stripe direction is obtained; a plurality of adjacent cumulative histogram values ​​of the window are used (step S1 is to obtain the cumulative histogram of all columns, and each calculation uses 5 adjacent columns, which are called adjacent columns, i.e., adjacent cumulative histogram values), and a new value is calculated according to the histogram inverse transformation and the average algorithm.

[0078] Assume that the original image has 100 columns. First, find the cumulative histogram of each column. Then, perform a histogram averaging operation every 5 columns. The first time: take columns 1 to 5, find the average, and use it as the new value of column 3; the second time: take columns 2 to 6, find the average, and use it as the new value of column 4; the third time: take columns 3 to 7, find the average, and use it as the new value of column 5; ..., the last time: take columns 96 to 100, find the average, and use it as the new value of column 98. The 5 columns taken each time are adjacent columns.

[0079] Using multiple adjacent cumulative histogram values ​​in the window, according to the inverse transformation of the histogram, and then using the improved average algorithm, a new value is calculated; the above results are combined with the original image and the original cumulative histogram to perform grayscale mapping, thereby obtaining the final image. The experimental results show that the application of the method of the present invention can maintain the effect of the original median histogram algorithm, and the process is more convenient to implement on the FPGA.

[0080] The embodiment of the present invention discloses an infrared image non-uniformity correction system based on FPGA, such as Figure 1 and Figure 2 As shown, the following modules are included: Module M1: Obtain the cumulative histogram value of the original image and obtain the original cumulative histogram value. Module M1 includes the following modules: Module M1.1: Obtain the histogram value of each row of the original image or each column of the original image.

[0081] Module M1.2: Calculate the cumulative histogram value of each row of the original image according to the histogram value of each row of the original image in module M1.1; or, calculate the cumulative histogram value of each column of the original image according to the histogram value of each column of the original image in module M1.1.

[0082] Module M2: Calculate new values ​​based on cumulative histogram values. Module M2 includes the following modules: Module M2.1: Group the cumulative histogram values ​​and determine the number of rows or columns of the cumulative histogram values ​​involved in the calculation.

[0083] Module M2.2: Obtain a new value by determining the cumulative histogram value of the number of rows or columns. Perform an inverse transformation on the cumulative histogram value of the determined number of rows or columns, and average the inverse transformed cumulative histogram value using the quartile method to obtain a new value. Sort the inverse transformed cumulative histogram values ​​to obtain sorted data.

[0084] The sorted data is divided equally by Q1, Q2 and Q3, where each part contains the sorted data divided equally; the calculation formula for obtaining Q1, Q2 and Q3 is:

[0085] Q3=Q2-1+Q1.

[0086] Among them, n represents the number of sorted data; Q1 represents the corresponding position of the quartile P0.25; Q2 represents the corresponding position of the quartile P0.5; Q3 represents the corresponding position of the quartile P0.75; P0.25 represents the data at the one-fourth position of the sorted data; P0.5 represents the data at the second-fourth position of the sorted data; P0.75 represents the data at the three-fourth position of the sorted data.

[0087] s=k*(P0.75-P0.25), where s represents the standard deviation of the sorted data; k represents the inverse of the standard normal interquartile range; the minimum estimated value of the sorted data is min=P0.75+1.5*s, and the maximum estimated value of the sorted data is max=P0.25-1.5*s; the new value is equal to (max+min) / 2.

[0088] Step S3: Mapping is performed based on the new value combined with the original image to obtain the final image. Grayscale mapping is performed based on the new value, the original cumulative histogram and the original image to obtain the final image;

[0089] The mapping relationship is:

[0090] final(i,j)=result(Chist(image(i,j)+1,j),j).

[0091] Among them, (i, j) represents the i-th row and j-th column, image represents the original image, Chist represents the original cumulative histogram image; result represents the results of the first four steps, and final represents the final image.

[0092] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0093] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A method for correcting infrared image non-uniformity based on FPGA, characterized in that: The steps include: Step S1: Obtain the cumulative histogram value of the original image; Step S2: Calculate a new value based on the cumulative histogram value; Step S3: Mapping is performed according to the new value combined with the original image to obtain a final image; The step S2 comprises the following steps: Step S2.1: Determine the number of rows or columns of the cumulative histogram values ​​involved in the calculation; Step S2.2: Obtain a new value by determining the cumulative histogram value of the number of rows or columns; In the step S2.2, the cumulative histogram values ​​of the determined number of rows or columns are subjected to an inverse transformation, and the inverse transformed cumulative histogram values ​​are averaged using the quartile method to obtain a new value; In step S2.2, the inverse transformed cumulative histogram values ​​are sorted to obtain sorted data; The sorted data is divided equally by Q1, Q2 and Q3, where each part contains the sorted data divided equally; the calculation formula for obtaining Q1, Q2 and Q3 is: Where n represents the number of sorted data; Q1 represents the position corresponding to the quartile P0.25; Q2 represents the position corresponding to the quartile P0.5; Q3 represents the position corresponding to the quartile P0.75; P0.25 represents the data at the quarter position of the sorted data; P0.5 represents the data at the second position of the sorted data; P0.75 represents the data at the third-fourth position of the sorted data; In step S2.2, s = k*(P0.75-P0.25); Where s represents the standard deviation of the sorted data; k represents the reciprocal of the standard normal interquartile range; The minimum estimated value of the sorted data is min=P0.75+1.5*s, the maximum estimated value of the sorted data is max=P0.25-1.5*s, and the new value is equal to (max+min) / 2.

2. The infrared image non-uniformity correction method based on FPGA according to claim 1 is characterized in that: The step S1 comprises the following steps: Step S1.1: Obtain the histogram value of each row or column of the original image; Step S1.2: Calculate the cumulative histogram value of each row of the original image according to the histogram value of each row of the original image in step S1.1; or, The cumulative histogram value of each column of the original image is calculated according to the histogram value of each column of the original image in step S1.

1.

3. An infrared image non-uniformity correction system based on FPGA, characterized in that: Includes the following modules: Module M1: Get the cumulative histogram value of the original image; Module M2: Calculate a new value based on the cumulative histogram value; Module M3: mapping the new value in combination with the original image to obtain a final image; The module M2 includes the following modules: Module M2.1: Determine the number of rows or columns of the cumulative histogram values ​​involved in the calculation; Module M2.2: Obtain new values ​​by determining the cumulative histogram value of the number of rows or columns; In the module M2.2, the cumulative histogram values ​​of the determined number of rows or columns are subjected to an inverse transformation, and the inverse transformed cumulative histogram values ​​are averaged using the quartile method to obtain a new value; In the module M2.2, the inverse transformed cumulative histogram values ​​are sorted to obtain sorted data; The sorted data is divided equally by Q1, Q2 and Q3, where each part contains the sorted data divided equally; the calculation formula for obtaining Q1, Q2 and Q3 is: Where n represents the number of sorted data; Q1 represents the position corresponding to the quartile P0.25; Q2 represents the position corresponding to the quartile P0.5; Q3 represents the position corresponding to the quartile P0.75; P0.25 represents the data at the quarter position of the sorted data; P0.5 represents the data at the second position of the sorted data; P0.75 represents the data at the third-fourth position of the sorted data; In the module M2.2, s = k*(P0.75-P0.25); Where s represents the standard deviation of the sorted data; k represents the reciprocal of the standard normal interquartile range; The minimum estimated value of the sorted data is min=P0.75+1.5*s, the maximum estimated value of the sorted data is max=P0.25-1.5*s, and the new value is equal to (max+min) / 2.

4. The FPGA-based infrared image non-uniformity correction system according to claim 3 is characterized in that: The module M1 includes the following modules: Module M1.1: Get the histogram value of each row or column of the original image; Module M1.2: Calculate the cumulative histogram value of each row of the original image according to the histogram value of each row of the original image in the module M1.1; or, The cumulative histogram value of each column of the original image is calculated according to the histogram value of each column of the original image of the module M1.1.

Images