FPGA-based infrared image data enhancement method

Abstract

The invention discloses an infrared image data enhancement method based on an FPGA, and relates to the field of image processing. Comprising the following steps: acquiring an original infrared video image output by an infrared imaging detector; calculating the cumulative distribution function of the gray level of the single-frame image in sequence, and calculating the upper and lower limits of the initial gray level of the single-frame image according to the preset upper and lower limits of the function integral value; according to the initial gray level upper and lower limits of the two adjacent frames of images, updating the gray level upper and lower limits of a single frame of image; and performing image enhancement processing on the original infrared video image according to the upper and lower limits of the gray level of the updated single-frame image to obtain an enhanced infrared video image. The method is used for solving the problems that a traditional image enhancement method is poor in universality and real-time performance and high in calculation complexity.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an infrared image data enhancement method based on FPGA. Background Art

[0002] In the field of image processing, especially infrared image processing, traditional image enhancement methods mainly include histogram equalization, contrast stretching, local contrast enhancement, etc. These methods can improve the image data effect to a certain extent, but because they usually rely too much on global statistical characteristics, they cannot effectively handle dynamically changing scenes, such as target motion, lighting changes, etc. In addition, traditional methods face the problems of high computational complexity and poor real-time performance when processing high-resolution and large-size infrared images.

[0003] Due to its strong parallel processing capability and high flexibility, FPGA has gradually become a common platform for high-speed, real-time image processing. However, existing FPGA-based infrared image enhancement methods are mostly limited to static image processing and lack effective processing methods for continuous video streams. Therefore, a method is needed that can not only improve the image contrast and detail expression capabilities, but also adapt to the infrared image enhancement needs in different scenarios, and have good versatility and real-time performance. Summary of the invention

[0004] The present invention aims to provide an infrared image data enhancement method based on FPGA to solve the problems of poor versatility, poor real-time performance and high computational complexity in traditional image enhancement methods.

[0005] To achieve the above object, the present invention adopts the following technical solution:

[0006] On the one hand, the present application provides an infrared image data enhancement method based on FPGA, comprising:

[0007] Acquire the original infrared video image output by the infrared imaging detector;

[0008] Calculate the cumulative distribution function of the gray level of the single frame image in sequence, and calculate the initial gray level upper and lower limits of the single frame image according to the preset upper and lower limits of the function integral value;

[0009] Updating the upper and lower limits of the grayscale of a single frame image according to the initial upper and lower limits of the grayscale of two adjacent frames of images;

[0010] According to the upper and lower limits of the grayscale of the updated single-frame image, the original infrared video image is subjected to image enhancement processing to obtain an enhanced infrared video image.

[0011] On the other hand, the present application also provides an infrared image data enhancement device based on FPGA, comprising:

[0012] An image acquisition module is used to acquire the original infrared video image output by the infrared imaging detector;

[0013] A grayscale calculation module is used to calculate the cumulative distribution function of the grayscale of a single frame image in sequence, and calculate the initial grayscale upper and lower limits of the single frame image according to the preset upper and lower limits of the function integral value;

[0014] A grayscale updating module, used for updating the grayscale upper and lower limits of a single frame image according to the initial grayscale upper and lower limits of two adjacent frames of images;

[0015] The image enhancement module is used to perform image enhancement processing on the original infrared video image according to the upper and lower limits of the grayscale level of the updated single-frame image to obtain an enhanced infrared video image.

[0016] On the other hand, the present application also provides an electronic device,

[0017] Processor; and

[0018] A memory arranged to store computer executable instructions which, when executed, cause the processor to perform the steps of a method as claimed in any one of the preceding claims.

[0019] Based on the above technical solution, this application can achieve the following technical effects:

[0020] The present invention discloses a highly real-time and highly versatile infrared image data enhancement method based on an FPGA platform, comprising: obtaining the original infrared video image output by an infrared imaging detector; calculating the cumulative distribution function of the grayscale of a single frame image in sequence, and calculating the initial grayscale upper and lower limits of the single frame image according to preset upper and lower limits of the function integral value; updating the grayscale upper and lower limits of the single frame image according to the initial grayscale upper and lower limits of two adjacent frames; performing image enhancement processing on the original infrared video image according to the updated grayscale upper and lower limits of the single frame image to obtain an enhanced infrared video image. The present invention fully considers the correlation of video images on the time axis, and designs a parameter optimization method to avoid abnormal enhancement effects caused by sudden changes in a frame of image, and has high real-time and highly versatile performance. At present, experiments have been carried out in multiple scenarios and have shown good enhancement effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flow chart of an infrared image data enhancement method based on FPGA provided in Example 1 of the present application;

[0022] Figure 2 This is a schematic diagram of an infrared image data enhancement device based on FPGA provided in Example 2 of the present application. DETAILED DESCRIPTION

[0023] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description and claims. It should be noted that the drawings are all in very simplified form and are not in precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0024] It should be noted that, in order to clearly explain the content of the present invention, the present invention specifically cites multiple embodiments to further illustrate different implementations of the present invention, wherein the multiple embodiments are enumerated rather than exhaustive. In addition, for the sake of brevity of explanation, the contents mentioned in the previous embodiments are often omitted in the subsequent embodiments. Therefore, the contents not mentioned in the subsequent embodiments can refer to the previous embodiments accordingly.

[0025] Example 1

[0026] Please refer to Figure 1 , Figure 1 The figure shows a flow chart of an infrared image data enhancement method based on FPGA provided in this embodiment. The method specifically comprises the following steps:

[0027] Step 102: Acquire the original infrared video image output by the infrared imaging detector.

[0028] It should be noted that one implementation of step 102 may be:

[0029] The infrared imaging detector is controlled by the front-end circuit and pre-processing FPGA software, and the analog signal output by the infrared imaging detector is converted into 14-bit infrared raw image data. After non-uniform correction processing, the infrared video image to be enhanced is obtained.

[0030] Step 104 , sequentially calculating the cumulative distribution function of the grayscale of the single frame image, and calculating the upper and lower limits of the initial grayscale of the single frame image according to the preset upper and lower limits of the function integral value.

[0031] It should be noted that one implementation of step 104 may be:

[0032] Initialize counter C1, count the gray level of each pixel of a single frame image, and obtain the gray level counting result of the single frame image;

[0033] Create a gray level array N1, perform cumulative distribution statistics on the counting results of the image gray levels, and obtain the cumulative distribution function of the gray level of a single frame image;

[0034] Traverse the grayscale array N1, obtain the grayscale corresponding to the preset upper and lower limits of the function integral value, and obtain the initial grayscale upper and lower limits of the single frame image. If there is no grayscale corresponding to the preset upper limit of the function integral value in the grayscale array, take the nearest grayscale upward as the initial grayscale upper limit; if there is no grayscale corresponding to the preset lower limit of the function integral value in the grayscale array, take the nearest grayscale downward as the initial grayscale lower limit.

[0035] Step 106: Update the upper and lower limits of the grayscale level of the single frame image according to the initial upper and lower limits of the grayscale level of two adjacent frames of images.

[0036] It should be noted that one implementation of step 106 may be:

[0037] Update the upper and lower limits of the grayscale of a single frame image using the following formula:

[0038] min_h n =a*min_h n +b*min_h n-1 (1)

[0039] max_h n =c*max_h n +d*max_h n-1 (2)

[0040] Among them, min_h n is the lower limit of the gray level of the n-th frame image, max_h n is the upper limit of the gray level of the nth frame image, min_h n-1 is the lower limit of the gray level of the n-1th frame image, max_h n-1 is the upper limit of the gray level of the n-1th frame image; in order to control the range of image enhancement gray level, adjustable parameters a, b, c, and d are set, a is the proportion of the gray level lower limit of the nth frame image in the process of updating the gray level lower limit, c is the proportion of the gray level upper limit of the nth frame image in the process of updating the gray level upper limit, b is the proportion of the gray level lower limit of the n-1th frame image after update in the process of updating the gray level lower limit, d is the proportion of the gray level upper limit of the n-1th frame image after update in the process of updating the gray level upper limit, a+b=1, c+d=1. Generally, parameters a and c are not less than 0.85, and parameters b and d are not greater than 0.15.

[0041] Step 108: Perform image enhancement processing on the original infrared video image according to the upper and lower limits of the grayscale of the updated single-frame image to obtain an enhanced infrared video image.

[0042] It should be noted that one implementation of step 108 may be:

[0043] In order to improve the real-time performance of data processing, since the image difference between two adjacent frames is usually not too large during image transmission, the calculation result of the previous frame can be used to process the current frame image. Specifically:

[0044] According to the upper and lower limits of the grayscale of the n-1th frame image, calculate the difference between the maximum grayscale values ​​of the n-1th frame image;

[0045] According to the difference between the grayscale maximum values ​​of the n-1th frame image, the first parameter of the n-1th frame image is calculated by the following formula:

[0046]

[0047] According to the difference between the grayscale maximum values ​​of the n-1th frame image and the first parameter of the n-1th frame image, the second parameter of the n-1th frame image is calculated by the following formula:

[0048]

[0049] According to the second parameter of the n-1th frame image and the lower limit of the gray level of the nth frame image, the enhanced nth frame infrared video image is obtained by the following formula:

[0050] gfl_out=K n-1 *(gfl_in-min_h)+b

[0051] Among them, gfl_out is the enhanced output n-th frame image, gfl_in is the original input n-th frame image; min_h is the lower limit of the grayscale level of the n-th frame image.

[0052] Further, a lookup table file is established, and the second parameter K is stored in the lookup table file, specifically:

[0053] Create a lookup table file. Since the input image width is 14 bits and the maximum grayscale difference is 16383, a lookup table file of size 16383 is created to cover all calculation results of parameter K. Parameter K can be quickly obtained based on the difference in the maximum grayscale values ​​of the n-1th frame image.

[0054] Furthermore, another lookup table file is established. Since the input image bit width is 14 bits and the maximum image grayscale difference is 16383, the lookup table only needs 16383 data at most to cover all the calculation results.

[0055] Based on this, using a lookup table to store pre-calculated results can avoid complex mathematical operations at runtime and reduce FPGA resource consumption.

[0056] In summary, the infrared image data enhancement method based on FPGA platform with high real-time performance and strong versatility disclosed in the present invention fully considers the correlation of video images on the time axis, and designs parameter optimization methods to avoid abnormal enhancement effects caused by sudden changes in one frame of image, and has high real-time performance and strong versatility. At present, it has been tested in multiple scenarios and has good enhancement effects.

[0057] Example 2

[0058] This embodiment provides an infrared image data enhancement method based on FPGA, including:

[0059] Step S1, controlling the infrared imaging detector through the front-end circuit and pre-processing FPGA software and converting the analog signal output by the detector into 14-bit infrared raw image data, and obtaining the infrared video image to be enhanced after non-uniform correction processing;

[0060] Step S2, calculating the cumulative distribution function cdf of the gray level of the entire frame image according to the single frame image, including:

[0061] S2.1 initializes a counter C1, counts the gray level of each pixel of the output valid image data, and increments the counter corresponding to the gray level address;

[0062] S2.2 creates a new array N1, calculates the cumulative distribution of grayscale according to the real-time data of counter C1, and adds the value of the previous entry to the next entry.

[0063] Step S3, calculate the upper and lower limits of the gray level of the nth frame image, and obtain the gray level min_h corresponding to cdf = 0.05. n and the gray level max_h corresponding to cdf = 0.95 n ,

[0064] In this embodiment, specifically:

[0065] According to step S2, the grayscale cumulative distribution of the whole frame image can be obtained by counting the array N1, and the grayscale values ​​corresponding to the upper and lower limits are obtained by traversing the array N1 according to the preset upper and lower limit values. Assuming that the cdf of the upper and lower limits are 0.05 and 0.95 respectively, when there is no grayscale level in the statistical result that can correspond to cdf=0.05, the nearest grayscale value is taken downward as the grayscale lower limit. Similarly, when there is no grayscale level that can correspond to cdf=0.95, the nearest grayscale value is taken upward as the grayscale upper limit.

[0066] Step S4, since the processed data is an infrared video image, the data is related on the time axis. Therefore, when processing data, it is necessary to fully consider the impact of historical data on the current frame image. In order to avoid sudden drastic changes in a frame image, which may cause abnormal grayscale of the frame image after processing, the control ratio of historical information on data is enhanced when calculating the highest and lowest grayscale levels starting from the first frame image, and at the same time, the upper and lower limits of the grayscale of the frame image are fully considered. The mathematical expression is defined as follows:

[0067] min_h n =a*min_h n +b*min_h n-1 (1)

[0068] max_h n =c*max_h n +d*max_h n-1 (2)

[0069] In the formula, in order to control the range of image enhancement grayscale, adjustable parameters a, b, c, and d are set, a is the proportion of the grayscale lower limit of the n-th frame image in the process of updating the grayscale lower limit, c is the proportion of the grayscale upper limit of the n-th frame image in the process of updating the grayscale upper limit, b is the proportion of the grayscale lower limit of the updated n-1-th frame image in the process of updating the grayscale lower limit, d is the proportion of the grayscale upper limit of the updated n-1-th frame image in the process of updating the grayscale upper limit, a+b=1, c+d=1, generally, parameters a and c are not less than 0.85, and parameters b and d are not greater than 0.15.

[0070] Using the serial processing flow, the calculation of the upper and lower limits of the grayscale of the current frame image is generally completed in the first clock cycle, and the update of the upper and lower limits of the grayscale of the image integrated with the historical information can be completed in the next clock cycle.

[0071] Step S5, based on the infrared video image obtained in step S1 and the upper and lower limits of the grayscale after each frame is updated obtained in step S4, an enhanced video image is obtained, and its mathematical expression is defined as follows:

[0072] gfl_out=K*(gfl_in-min_h)+b (3)

[0073]

[0074] In formula (3) It represents the output image after image enhancement processing, and gfl_in represents the 14-bit image input after non-uniform correction.

[0075] Step S5 includes:

[0076] S5.1 Calculation of the difference between the maximum gray values;

[0077] S5.2 Create a lookup table file coe1 to obtain the calculation result of parameter D. Since the input image bit width is 14 bits and the maximum image grayscale difference is 16383, the lookup table only needs 16383 data at most to cover all the calculation results.

[0078] S5.3 creates a lookup table file coe2. The absolute values ​​of the maximum values ​​used when calculating parameters K and D according to formulas (4) and (5) are completely consistent. Therefore, after converting formula (4) and combining it with formula (5), a lookup table file with a size of 16383 is created. According to the results obtained in S5.1, parameter K can be obtained.

[0079] S5.4 In order to improve the real-time performance of data processing, since the image difference between two adjacent frames is usually not too large during image transmission, the calculation result of the previous frame can be used to process the current frame image;

[0080] The present embodiment is specifically as follows:

[0081] At the end of the effective data transmission of the n-1th frame image, the parameter K n-1 and D n-1 Preparations are complete. When the effective data of the nth frame image starts to be transmitted, there is formula (6):

[0082] gfl_out(i)=K n-1 *[gfl_in(i)-min_h]+b (6)

[0083] In formula (6), i means that the current data is the i-th valid data in the current frame image.

[0084] In summary, the infrared image data enhancement method based on FPGA platform with high real-time performance and strong versatility disclosed in the present invention fully considers the correlation of video images on the time axis, and designs parameter optimization methods to avoid abnormal enhancement effects caused by sudden changes in one frame of image, and has high real-time performance and strong versatility. At present, it has been tested in multiple scenarios and has good enhancement effects.

[0085] Example 3

[0086] Please refer to Figure 2 , Figure 2 FIG. 1 is a schematic diagram of an infrared image data enhancement device based on FPGA provided in this embodiment. The device specifically comprises:

[0087] The image acquisition module 302 is used to acquire the original infrared video image output by the infrared imaging detector;

[0088] Gray level calculation module 304, used to calculate the cumulative distribution function of the gray level of the single frame image in sequence, and calculate the initial gray level upper and lower limits of the single frame image according to the preset upper and lower limits of the function integral value;

[0089] A grayscale updating module 306, configured to update the grayscale upper and lower limits of a single frame image according to the initial grayscale upper and lower limits of two adjacent frames of image;

[0090] The image enhancement module 308 is used to perform image enhancement processing on the original infrared video image according to the upper and lower limits of the grayscale of the updated single-frame image to obtain an enhanced infrared video image.

[0091] Optionally, the device further includes a table lookup file module, specifically used for:

[0092] A lookup table file is established, and the first parameter and the second parameter are stored in the lookup table file.

[0093] Based on this, using a lookup table to store pre-calculated results can avoid complex mathematical operations at runtime and reduce FPGA resource consumption.

[0094] In summary, the infrared image data enhancement method based on FPGA platform with high real-time performance and strong versatility disclosed in the present invention fully considers the correlation of video images on the time axis, and designs parameter optimization methods to avoid abnormal enhancement effects caused by sudden changes in one frame of image, and has high real-time performance and strong versatility. At present, it has been tested in multiple scenarios and has good enhancement effects.

[0095] Example 4

[0096] In another feasible embodiment, this embodiment provides a device for infrared image data enhancement based on FPGA, and the device may specifically include:

[0097] A processor; and a memory arranged to store computer executable instructions, which, when executed, cause the processor to perform the steps in any of the above method embodiments.

[0098] Example 5

[0099] In another feasible embodiment, this embodiment provides a storage medium for infrared image data enhancement based on FPGA, and the storage medium may specifically include:

[0100] The storage medium stores a processing program for infrared image data enhancement based on FPGA, and when the processing program for infrared image data enhancement based on FPGA is executed by a processor, the steps in any of the above method embodiments are implemented.

[0101] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application can be modified and varied in various ways. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for infrared image data enhancement based on FPGA, characterized in that: include: Acquire the original infrared video image output by the infrared imaging detector; Calculate the cumulative distribution function of the gray level of the single frame image in sequence, and calculate the initial gray level upper and lower limits of the single frame image according to the preset upper and lower limits of the function integral value; Updating the upper and lower limits of the grayscale of a single frame image according to the initial upper and lower limits of the grayscale of two adjacent frames of images; According to the upper and lower limits of the grayscale of the updated single-frame image, the original infrared video image is subjected to image enhancement processing to obtain an enhanced infrared video image.

2. The method according to claim 1, characterized in that The step of sequentially calculating the cumulative distribution function of the grayscale level of the single-frame image and calculating the initial grayscale upper and lower limits of the single-frame image according to the preset upper and lower limits of the function integral value includes: Initialize the counter, count the gray level of each pixel of the single frame image, and obtain the counting result of the gray level of the single frame image; Creating a grayscale array, performing cumulative distribution statistics on the counting results of the grayscale of the image, and obtaining a cumulative distribution function of the grayscale of a single frame image; The grayscale array is traversed to obtain grayscales corresponding to preset upper and lower limits of the function integral value, so as to obtain initial grayscale upper and lower limits of the single-frame image.

3. The method according to claim 2, characterized in that The traversing the grayscale array, obtaining the grayscale corresponding to the preset upper and lower limits of the function integral value, and obtaining the initial grayscale upper and lower limits of the single-frame image, includes: If there is no gray level corresponding to the preset upper limit of the function integral value in the gray level array, the nearest gray level is taken upward as the initial gray level upper limit; If there is no gray level corresponding to the preset lower limit of the function integral value in the gray level array, the nearest gray level is taken downward as the initial gray level lower limit.

4. The method according to claim 1, characterized in that: The updating of the upper and lower limits of the grayscale level of a single frame image according to the initial upper and lower limits of the grayscale level of two adjacent frames of images includes: Update the upper and lower limits of the grayscale of a single frame image using the following formula: min_h n =a*min_h n +b*min_h n-1 max_h n =c*max_h n +d*max_h n-1 Among them, min_h n is the lower limit of the gray level of the n-th frame image, max_h n is the upper limit of the gray level of the nth frame image, min_h n-1 is the lower limit of the gray level of the n-1th frame image, max_h n-1 is the upper limit of the grayscale level of the n-1th frame image; in order to control the range of image enhancement grayscale, adjustable parameters a, b, c, and d are set, where a is the proportion of the grayscale lower limit of the n-th frame image in the process of updating the grayscale lower limit, c is the proportion of the grayscale upper limit of the n-th frame image in the process of updating the grayscale upper limit, b is the proportion of the grayscale lower limit of the n-1th frame image after update in the process of updating the grayscale lower limit, d is the proportion of the grayscale upper limit of the n-1th frame image after update in the process of updating the grayscale upper limit, a+b=1, c+d=1.

5. The method according to claim 1, characterized in that The method of performing image enhancement processing on the original infrared video image according to the upper and lower limits of the grayscale level of the updated single-frame image to obtain an enhanced infrared video image includes: According to the upper and lower limits of the grayscale of the n-1th frame image, calculate the difference between the maximum grayscale values ​​of the n-1th frame image; According to the difference between the grayscale maximum values ​​of the n-1th frame image and the grayscale lower limit of the nth frame image, the nth frame original infrared video image is subjected to image enhancement processing to obtain an enhanced nth frame infrared video image.

6. The method according to claim 5, characterized in that The image enhancement process is performed on the original infrared video image of the nth frame according to the difference between the grayscale maximum values ​​of the n-1th frame image and the grayscale lower limit of the nth frame image to obtain the enhanced infrared video image of the nth frame, including: According to the difference between the grayscale maximum values ​​of the n-1th frame image, the first parameter of the n-1th frame image is calculated by the following formula: According to the difference between the grayscale maximum values ​​of the n-1th frame image and the first parameter of the n-1th frame image, the second parameter of the n-1th frame image is calculated by the following formula: According to the second parameter of the n-1th frame image and the lower limit of the gray level of the nth frame image, the enhanced nth frame infrared video image is obtained by the following formula: gfl_out=K n-1 *(gfl_in-min_h)+b Among them, gfl_out is the enhanced output n-th frame image, gfl_in is the original input n-th frame image; min_h is the lower limit of the grayscale level of the n-th frame image.

7. The method according to claim 6, characterized in that After calculating the second parameter of the n-1th frame image by the following formula according to the difference between the grayscale maximum values ​​of the n-1th frame image and the first parameter of the n-1th frame image, the method further includes: A lookup table file is established, and the first parameter and the second parameter are stored in the lookup table file.

8. The method according to claim 1, characterized in that After the original infrared video image output by the infrared imaging detector is obtained, the method further includes: The infrared imaging detector is controlled through the front-end circuit and pre-processing FPGA software, and the analog signal output by the infrared imaging detector is converted into 14-bit infrared raw image data; The 14-bit infrared raw image data is subjected to data splicing, sorting and non-uniform correction processing.

9. An infrared image data enhancement device based on FPGA, characterized in that: include: An image acquisition module is used to acquire the original infrared video image output by the infrared imaging detector; A grayscale calculation module is used to calculate the cumulative distribution function of the grayscale of a single frame image in sequence, and calculate the initial grayscale upper and lower limits of the single frame image according to the preset upper and lower limits of the function integral value; A grayscale updating module, used for updating the grayscale upper and lower limits of a single frame image according to the initial grayscale upper and lower limits of two adjacent frames of images; The image enhancement module is used to perform image enhancement processing on the original infrared video image according to the upper and lower limits of the grayscale level of the updated single-frame image to obtain an enhanced infrared video image.

10. An electronic device, characterized in that: include: processor; as well as A memory arranged to store computer executable instructions which, when executed, cause the processor to perform the steps of the method as claimed in any one of claims 1 to 8.

Images