FPGA-based low-light image real-time enhancement method

Abstract

According to the FPGA-based low-light-level image real-time enhancement method provided by the invention, by utilizing the characteristics of low power consumption, high parallel computing degree and flexible programming of the FPGA, real-time enhancement is carried out on an image acquired by an image sensor in a low-light-level environment, and night target information which cannot be distinguished by human eyes is changed into a visual image. The method comprises the steps of Gaussian filtering, Gamma transformation, image blocking, gray level histogram cutting and reconstruction, histogram equalization and bilinear interpolation on an FPGA (Field Programmable Gate Array). According to the method, the processing speed is higher, the root-mean-square error of the enhanced image is obviously reduced, the peak signal-to-noise ratio is obviously improved, and the structural similarity is doubled.

Description

Technical Field

[0001] This application relates to the field of digital image processing technology, and particularly relates to a real-time low-light image enhancement method based on FPGA. Background Art

[0002] Low-light night vision technology uses optoelectronic imaging devices to detect the radiation or reflection information of optical signals of targets at night, and converts the information of night targets that cannot be distinguished by the human eye into visible images through the acquisition, processing, and display of photodetectors and imaging devices. With the continuous iteration and development of technology, low-light night vision technology has been widely used in military, security, agriculture, automotive and other fields.

[0003] When the illuminance is lower than 0.1 Lux, the noise in the low-light image is mainly Gaussian noise at this time. Most of the noise in the low-light image can be effectively removed through Gaussian filtering. Gamma transformation is a commonly used technology in digital image processing, mainly used to adjust the brightness of the image, so that the gray value of the output image has an exponential relationship with the gray value of the input image, increasing the proportion of the dark part and the light part in the image signal, and improving the image brightness. CLAHE uses a threshold to clip the gray histogram of the image, and the peak part of the histogram is trimmed and assigned to other gray levels, so as to limit the slope of the cumulative histogram function of the image; it can effectively control the over-enhancement of the contrast, so as to better process low-contrast images with noise, and solve the problems existing in the classical histogram equalization.

[0004] However, most of the implementation methods of the above algorithms are currently implemented on the host computer, and there is a certain delay compared with the real-time picture collected by the image sensor, which is not conducive to obtaining night information faster. Summary of the Invention

[0005] The present invention proposes a real-time low-light image enhancement method based on FPGA, which effectively improves the image processing speed by using the high parallel computing degree and fastness of FPGA, and can solve the problem of picture delay existing in the prior art.

[0006] The technical solution to implement the present invention is: programming on the FPGA to implement the image processing flow of Gaussian filtering + Gamma transformation + CLAHE, and the specific steps are as follows:

[0007] Step 1: Perform Gaussian filtering on the original image collected by the image sensor;

[0008] Step 2: Perform Gamma transformation;

[0009] Step 3: Divide the image into blocks;

[0010] Step 4: Statistically calculate the gray histogram of each sub-block of the image;

[0011] Step 5: Crop the grayscale histogram of each image sub-block to limit the peak value of the grayscale histogram of each sub-block;

[0012] Step 6: Equalize the grayscale histogram of each image sub-block;

[0013] Step 7: Perform bilinear interpolation between adjacent sub-blocks to eliminate the boundary effect between each sub-block.

[0014] Preferably, the specific process of performing Gaussian filtering on the original image collected by the image sensor is as follows:

[0015] Step 1.1: Generate a 3×3 image window and perform a sliding window operation. Use two FIFOs to cache the row data of the image. The depth of the FIFO is greater than the number of image columns. Cache two rows of data in a head-to-tail connection form. When the third row of data arrives, read the data in the two FIFOs simultaneously, so that three rows of data can be output simultaneously. At this time, the data read from the last FIFO is the data of the first row.

[0016] Step 1.2: Determine the convolution of the Gaussian template and the image window. In the 3×3 Gaussian template, assume that the coordinates of the center point are (0, 0), and calculate the corresponding Gaussian template according to the two-dimensional Gaussian function G(x, y) and the coordinate values. The two-dimensional Gaussian function is:

[0017]

[0018] σ is the standard deviation of the Gaussian distribution, which determines the width of the Gaussian function (i.e., the distribution degree of pixel points near the center point (mean value)). The larger the σ value, the more blurred the filter, the stronger the noise removal ability, but more image details may be lost.

[0019] Preferably, the specific steps to implement Gamma transformation are as follows:

[0020] Step 2.1: Generate the corresponding lookup table code in Verilog through MATLAB code. The formula for Gamma transformation is:

[0021]

[0022] where n is the number of bits of the image pixel, V in is the input grayscale value, and the exponential coefficient γ is the coefficient of Gamma transformation. After Gamma transformation, if γ < 1, the details in the dark part can be enhanced and the details in the bright part can be compressed; if γ > 1, the details in the bright part are enhanced and the details in the dark part are compressed.

[0023] Step 2.2: Instantiate the code generated in Step 2.1. The core of this code is the case statement. The input data is the original image grayscale, and the output data is the transformed grayscale value.

[0024] Preferably, the specific steps for implementing image block division are as follows:

[0025] Step 3.1: Determine the horizontal and vertical coordinates of the input image pixel points and record them through two counters;

[0026] Step 3.2: Determine which sub-block the currently transmitted data is located in based on the horizontal and vertical coordinates, pull up the corresponding sub-block enable signal part_en[i] and sub-block data signal part_data_in[i] (the data format is a two-dimensional array), and pass it into the corresponding histogram statistics module. The number of sub-blocks is determined by the parameters PART_X and PART_Y. The parameter PART_X represents the number of equal divisions in the horizontal direction, and the parameter PART_Y represents the number of equal divisions in the vertical direction.

[0027] Preferably, the specific steps for implementing the grayscale histogram statistics of each sub-block of the image are as follows:

[0028] Step 4.1: Instantiate RAM1 to statistically calculate the grayscale histogram of the image sub-block. The read and write addresses of RAM1 correspond to the grayscale values of each pixel in the image sub-block, and the data stored at each address is the number of pixels with the corresponding grayscale value. When the rising edge of the on-site valid signal is effective, clear RAM1.

[0029] Step 4.2: RAM1 is used as a dual-port RAM. Port A is responsible for writing the number of pixels of the currently statistically calculated grayscale level, and port B is responsible for reading out the number of pixels his_doutb statistically calculated for the current pixel grayscale value. During the process of statistical histogram calculation, the most important thing is to calculate the number of pixels to be accumulated each time, pix_same_cnt. The image data is clocked. If the adjacent grayscale values are different, the number of pixels to be accumulated, pix_same_cnt, is 1. When pixels with the same grayscale value appear continuously, the consecutive number needs to be continuously incremented to calculate the consecutive number pix_same_cnt. When the adjacent data is different, set pix_same_cnt to 1;

[0030] Step 4.3: Add pix_same_cnt to the number of pixels his_doutb of the current pixel grayscale value read out from port B. The result is the number of pixels of the current grayscale value after statistics. At the same time, pull up the write enable signal of port A and write it into the corresponding address. When the last line of image data is input, the statistical work of the grayscale histogram of the image by RAM1 is completed.

[0031] Preferably, the specific process of cropping the grayscale histogram of each image sub-block is as follows:

[0032] Step 5.1: Select an appropriate threshold CLIP_LIMIT, and its calculation formula is:

[0033]

[0034] Where HEIGHT and WIDTH are the number of rows and columns of the image respectively, PART_X × PART_Y represents the number of sub-blocks, and n is the number of bits of the image pixels.

[0035] Step 5.2: Read out the data stored in RAM1, and judge whether its size is greater than the threshold CLIP_LIMIT. If it is greater, subtract the threshold, and accumulate the obtained results to get the total number of pixels sum_excess greater than the threshold. After reading out all the data, divide sum_excess by the number of gray levels 2 n , to get the average number of pixels exceeding each gray level bin_averate, and subtract bin_averate from the threshold CLIP_LIMIT to get the upper limit upper of the new gray histogram.

[0036] Step 5.3: Read out the number of pixels of each gray level stored in RAM1 again, and judge its size. If it is greater than the threshold CLIP_LIMIT, assign clip_dina to CLIP_LIMIT and sum_excess remains unchanged; if it is greater than the threshold CLIP_LIMIT and less than the upper limit upper, assign clip_dina to CLIP_LIMIT and subtract (CLIP_LIMIT - his_doutb) from sum_excess; if it is less than the upper limit, increase pixel_num by bin_averate. Assign it to clip_dina and subtract bin_averate from sum_excess. Store the obtained clip_dina in RAM2 in the order of gray levels. After completion, shift sum_excess to the right by the number of bits of the pixel data and add 1 to prevent it from being 0, and assign the result to step_size.

[0037] Step 5.4: Next, read out the data stored in RAM2 in the order of gray values, add step_size to it and assign it to clip_dina, and at the same time subtract step_size from sum_excess, and write clip_dina into the corresponding address of RAM2. When sum_excess is less than zero, stop reading the data in RAM2, and the redistribution of the gray histogram is completed. At this time, the data stored in RAM2 is the histogram of the cropped and redistributed image sub-blocks.

[0038] Preferably, the specific process of equalizing the gray histogram after cropping and reconstructing each image sub-block is as follows:

[0039] Step 6.1: The definition of the cumulative distribution function (CDF) is the number of pixels less than or equal to a certain gray value. The calculation formula is:

[0040]

[0041] where h(r i ) is the number of pixels with gray value r i , and k is the gray value of the image pixel.

[0042] The read address of port b of RAM2 counts from 0 to the maximum gray level, reads the data at each address in sequence, and accumulates the number of pixels for each read gray level.

[0043] Step 6.2: By performing a linear transformation on the CDF, map it to the target gray value range [0, L - 1]. The target gray value s k corresponds to the original gray value r k and the mapping relationship is:

[0044]

[0045] where SUM is the total number of pixels in the image sub - block, and [·] represents the floor operation.

[0046] Multiply the accumulated number of pixels by the maximum gray level (2 n - 1), divide by the total number of pixels in the image sub - block, write the calculated target gray value into RAM3, use the gray value of the original pixel as its write address, and the converted target gray value as the stored data.

[0047] Preferably, perform bilinear interpolation between adjacent sub - blocks to eliminate the boundary effect between sub - blocks. The specific steps are as follows:

[0048] Step 7.1: Determine the position of the interpolation point. Suppose there is an input image, and the goal is to calculate a new pixel value based on the pixel points of the image at the coordinate position (x, y). First, find the four neighboring pixel points around this coordinate point. Usually, the positions of these four pixel points are Q 11 (x1, y1), Q 12 (x1, y2), Q 21 (x2, y1) and Q 22 (x2, y2). At this time, the calculation formula for bilinear interpolation is as follows:

[0049]

[0050] In image processing, usually set the coordinates of (x1, y1) in the formula to (0, 0). At this time, the above formula can be simplified to:

[0051]

[0052] From the above introduction of bilinear interpolation, if (x1, y1) is (0, 0), it is to find the gray values at each position inside a window of size x2×y2.

[0053] Step 7.2: Perform interpolation calculation. When performing bilinear interpolation calculation, the values of x2 and y2 are related to which sub-block the pixel is in. For the horizontal and vertical coordinates (x, y) inside the window, they need to be obtained according to the horizontal and vertical coordinates of the pixel in the image sub-block. At this time, f(x1, y1), f(x2, y1), f(x1, y2), and f(x2, y2) are the gray values after histogram equalization inside different corresponding sub-blocks. Substituting them into the formula, the finally enhanced gray value is obtained.

[0054] Compared with the prior art, the significant advantages of the present invention are: (1) Utilizing the advantages of high parallel computing and high clock frequency of FPGA, a faster processing speed is achieved, reducing the delay on the screen. In addition, FPGA has the characteristics of small volume and low energy consumption, which is conducive to applying the image enhancement method to miniaturized portable devices; (2) The low-light image enhancement algorithm of Gaussian filtering + Gamma transformation + CLAHE can clearly identify the outline and shape of an object at an illuminance of 0.001 Lux, adding more details. The results show that the root mean square error (RMSE) is reduced by 25%, the peak signal-to-noise ratio (PSNR) is increased by 15% on average, and the structural similarity (SSIM) is doubled.

[0055] The following further describes the present invention in detail with reference to the accompanying drawings. Description of the Drawings

[0056] Figure 1 It is the flowchart of the enhancement algorithm of the present invention.

[0057] Figure 2 It is the flowchart of the FIFO cache outputting three lines of data diagram of the present invention.

[0058] Figure 3 It is the flowchart when the image of the present invention is divided into blocks.

[0059] Figure 4 It is the image coordinate diagram of bilinear interpolation.

[0060] Figure 5 It is the region division when the present invention performs bilinear interpolation.

[0061] Figure 6 It is the effect of the present invention on image enhancement before and after under different low illuminance conditions. Detailed Embodiment

[0062] To make the purpose, technical solution and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0063] First, the embodiments of the present application will be introduced below in conjunction with the accompanying drawings.

[0064] The concept of the present invention is based on the characteristics of FPGA, such as low power consumption, high parallel computing degree, and flexible programming. It performs real-time enhancement on the images collected by the image sensor in low-light environments, turning the night target information that cannot be distinguished by the human eye into visible images. The algorithm flow is as Figure 1 shown. The specific steps are as follows:

[0065] Step 1: Perform Gaussian filtering on the original image collected by the image sensor.

[0066] Step 1.1: Generate a 3×3 image window and perform a sliding window operation. Use two FIFOs to cache the row data of the image. The depth of the FIFO is greater than the number of image columns. Cache two rows of data in a head-to-tail connection form. When the third row of data arrives, read the data in the two FIFOs simultaneously, so that three rows of data can be output simultaneously. At this time, the data read from the last FIFO is the data of the first row, as Figure 2 shown.

[0067] Step 1.2: Determine the convolution of the Gaussian template and the image window. In the 3×3 Gaussian template, assuming the coordinates of the center point are (0, 0), calculate the corresponding Gaussian template according to the two-dimensional Gaussian function G(x, y) and the coordinate values. The two-dimensional Gaussian function is:

[0068]

[0069] σ is the standard deviation of the Gaussian distribution, which determines the width of the Gaussian function (i.e., the distribution degree of pixel points near the center point (mean value)). The larger the σ value, the more blurred the filter, the stronger the noise removal ability, but more image details may be lost.

[0070] Step 2: Perform Gamma transformation.

[0071] Step 2.1: Generate the corresponding lookup table code in Verilog through MATLAB code. The formula for Gamma transformation is:

[0072]

[0073] where n is the number of bits of the image pixel, V in is the input grayscale value, and the exponential coefficient γ is the coefficient of Gamma transformation. After Gamma transformation, if γ < 1, the details of the dark part can be enhanced and the details of the bright part can be compressed; if γ > 1, the details of the bright part are enhanced and the details of the dark part are compressed.

[0074] Step 2.2: Instantiate the code generated in Step 2.1. The core of this code is the case statement, with the input data being the original image grayscale and the output data being the transformed grayscale value.

[0075] Step 3: Segment the image.

[0076] Step 3.1: Determine the horizontal and vertical coordinates of the input image pixels and record them using two counters.

[0077] Step 3.2: Determine which sub-block the currently transmitted data is located in based on the horizontal and vertical coordinates, pull up the corresponding sub-block enable signal part_en[i] and sub-block data signal part_data_in[i] (data format is a two-dimensional array), and pass it into the corresponding histogram statistics module, as Figure 3 shown. The number of sub-blocks is determined by the parameters PART_X and PART_Y. The parameter PART_X represents the number of equal divisions horizontally, and the parameter PART_Y represents the number of equal divisions vertically.

[0078] Step 4: Statistically analyze the grayscale histogram of each sub-block of the image.

[0079] Step 4.1: Instantiate RAM1 to statistically analyze the grayscale histogram of the image sub-block. The read and write addresses of RAM1 correspond to the grayscale values of each pixel in the image sub-block, and the data stored at each address is the number of pixels with the corresponding grayscale value. Clear RAM1 when the rising edge of the on-site valid signal is effective.

[0080] Step 4.2: RAM1 is a dual-port RAM. Port A is responsible for writing the number of pixels of the currently statistically analyzed grayscale level, and port B is responsible for reading out the number of pixels his_doutb statistically analyzed for the current pixel grayscale value. During the process of statistically analyzing the histogram, the most important thing is to calculate the number pix_same_cnt to be accumulated each time. Beat the image data. If the adjacent grayscale values are different, the number pix_same_cnt to be accumulated is 1. When pixels with the same grayscale value appear continuously, continuously increment by one to count and calculate the consecutive number pix_same_cnt. When the adjacent data is different, set pix_same_cnt to 1.

[0081] Step 4.3: Add pix_same_cnt to the number of pixels his_doutb of the current pixel grayscale value read out from port B. The result is the number of pixels of the current grayscale value after statistical analysis. At the same time, pull up the write enable signal of port A and write it into the corresponding address. When the last line of image data is input, the statistical work of the grayscale histogram of the image by RAM1 is completed.

[0082] Step 5: Crop the grayscale histogram of each image sub-block.

[0083] Step 5.1: Select an appropriate threshold value CLIP_LIMIT, and its calculation formula is:

[0084]

[0085] where HEIGHT and WIDTH are the number of rows and columns of the image respectively, PART_X × PART_Y represents the number of sub-blocks, and n is the number of bits of the image pixels.

[0086] Step 5.2: Read out the data stored in RAM1, and determine whether its size is greater than the threshold value CLIP_LIMIT. If it is greater, subtract the threshold value and accumulate the obtained results to get the total number of pixels sum_excess that are greater than the threshold value. After reading out all the data, divide sum_excess by the number of gray levels 2 n , to obtain the average number of pixels exceeding each gray level bin_averate. Subtract bin_averate from the threshold value CLIP_LIMIT to get the upper limit upper of the new gray histogram.

[0087] Step 5.3: Read out the number of pixels of each gray level stored in RAM1 again, and judge its size. If it is greater than the threshold value CLIP_LIMIT, assign clip_dina to CLIP_LIMIT and keep sum_excess unchanged; if it is greater than the threshold value CLIP_LIMIT and less than the upper limit upper, assign clip_dina to CLIP_LIMIT and subtract (CLIP_LIMIT - his_doutb) from sum_excess; if it is less than the upper limit, increase pixel_num by bin_averate. Assign the result to clip_dina and subtract bin_averate from sum_excess. Store the obtained clip_dina in RAM2 in the order of gray levels. After completion, shift sum_excess to the right by the number of bits of the pixel data and add 1 to prevent it from being 0, and assign the result to step_size.

[0088] Step 5.4: Next, read out the data stored in RAM2 in the order of gray values, add step_size to it and assign the result to clip_dina, and at the same time subtract step_size from sum_excess. Write clip_dina into the corresponding address of RAM2. When sum_excess is less than zero, stop reading out the data in RAM2, and the redistribution of the gray histogram is completed. At this time, the data stored in RAM2 is the histogram of the cropped and redistributed image sub-blocks.

[0089] Step 6: Equalize the gray histogram of each cropped and reconstructed image sub-block.

[0090] Step 6.1: The cumulative distribution function (CDF) is defined as the number of pixels less than or equal to a certain gray value. The calculation formula is:

[0091]

[0092] where h(r i ) is the number of pixels with gray value r i , and k is the gray value of the image pixel.

[0093] The read address of port b of RAM2 counts from 0 to the maximum gray level, reads the data of each address in sequence, and accumulates the number of pixels of each read gray value.

[0094] Step 6.2: By linearly transforming the CDF, map it to the target gray value range [0, L - 1]. The mapping relationship between the target gray value s k corresponding to the original gray value r k is:

[0095]

[0096] where SUM is the total number of pixels in the image sub-block, and [·] represents the floor operation.

[0097] Multiply the accumulated number of pixels by the maximum gray level (2 n -1), divide by the total number of pixels in the image sub-block, write the calculated target gray value into RAM3, use the gray value of the original pixel as its write address, and the converted target gray value as the stored data.

[0098] Step 7: Perform bilinear interpolation between adjacent sub-blocks to eliminate the boundary effect between sub-blocks.

[0099] Step 7.1: Determine the position of the interpolation point. Suppose there is an input image, and the goal is to calculate a new pixel value based on the pixel points of the image at the coordinate position (x, y). First, find the four neighboring pixel points around this coordinate point. Usually, the positions of these four pixel points are Q 11 (x1, y1), Q 12 (x1, y2), Q 21 (x2, y1) and Q 22 (x2, y2). As Figure 4 shown. At this time, the calculation formula for bilinear interpolation is as follows:

[0100]

[0101]

[0102] In image processing, the coordinates of (x1, y1) in the formula are usually set to (0, 0), and at this time the above formula can be simplified to:

[0103]

[0104] From the above introduction to bilinear interpolation, if (x1, y1) is (0, 0), it is to find the gray values of each position inside a window of size x2×y2.

[0105] Step 7.2: Perform interpolation calculation. As Figure 5 shown, the dotted line divides each sub-block into four equal parts. The part filled with stripes is the part containing the sub-block boundary. To eliminate the boundary effect, the pixels in the horizontal stripe and vertical stripe regions need to perform bilinear interpolation according to the two adjacent sub-blocks where they are located, and the pixels in the oblique stripe region perform bilinear interpolation according to the four adjacent sub-blocks where they are located.

[0106] When performing bilinear interpolation calculation, the values of x2 and y2 are related to the regional color of the pixel in the entire image. In addition, it is necessary to calculate according to the horizontal and vertical coordinates of the pixel in the entire image to obtain the horizontal and vertical coordinates (x, y) inside the sub-block. At this time, f(x1, y1), f(x2, y1), f(x1, y2), and f(x2, y2) are the gray values after histogram equalization inside different sub-blocks. Substituting them into the formula, the finally enhanced gray value is obtained.

[0107] Select a dark room with an electromagnetic isolation layer and controllable light intensity as the test location. At illuminances of 0.1 Lux, 0.01 Lux, and 0.001 Lux, the final presented effects are as Figure 6 shown. The left side is the original image under different illuminances, and the right side is the image after enhancement processing.

[0108] The embodiments of the present application described above do not constitute a limitation on the protection scope of the present application.

Claims

1. A real-time low-light image enhancement method based on FPGA, characterized in that: The method comprises the following steps: Step 1: Perform Gaussian filtering on the original image collected by the image sensor; Step 2: Perform Gamma transformation; Step 3: Divide the image into blocks; Step 4: Count the grayscale histogram of each sub-block of the image; Step 5: Crop the grayscale histogram of each image sub-block to limit the peak value of the grayscale histogram of each sub-block; Step 6: Equalize the grayscale histogram of each image sub-block; Step 7: Perform bilinear interpolation between adjacent sub-blocks to eliminate the boundary effect between each sub-block.

2. The FPGA-based real-time low-light image enhancement method according to claim 1, characterized in that: Perform Gaussian filtering on the original image collected by the image sensor, including: Step 1.1: Generate a 3×3 image window and perform a sliding window operation: Use two FIFOs to cache the row data of the image. The FIFO depth is greater than the number of image columns. Two rows of data are cached in a head-to-tail manner. When the third row of data arrives, the data in the two FIFOs is read out at the same time, that is, three rows of data are output at the same time. At this time, the last FIFO reads the data of the first row; Step 1.2: Determine the convolution of the Gaussian template and the image window: In the 3×3 Gaussian template, assume that the coordinates of the center point are (0, 0), and calculate the corresponding Gaussian template based on the two-dimensional Gaussian function G(x, y) and the coordinate value; the two-dimensional Gaussian function is: σ is the standard deviation of the Gaussian distribution, which determines the width of the Gaussian function, that is, the distribution of pixels around the center point).

3. The real-time low-light image enhancement method based on FPGA according to claim 1, characterized in that: Perform Gamma transformation, including: Step 2.1: Generate the corresponding Verilog lookup table code through MATLAB code; the Gamma transformation method is: Where n is the number of bits of the image pixel, V in is the input grayscale value, and the exponential coefficient γ is the coefficient of the Gamma transformation. After the Gamma transformation, if γ<1, the dark details can be enhanced and the bright details can be compressed; if γ>1, the bright details are enhanced and the dark details are compressed. Step 2.2: Instantiate the code generated in step 2.1; the core of the code is the case statement, the input data is the original image grayscale, and the output data is the transformed grayscale value.

4. The method for real-time enhancement of low-light-level images based on FPGA according to claim 1, characterized in that: Perform image segmentation, including: Step 3.1: Determine the horizontal and vertical coordinates of the pixel points of the input image and record them through two counters; Step 3.2: Use the horizontal and vertical coordinates to determine which sub-block the currently transmitted data is located in, pull up the corresponding sub-block enable signal part_en[i] and sub-block data signal part_data_in[i], that is, the data format is a two-dimensional array, and pass it into the corresponding histogram statistics module; the number of sub-blocks is determined by the parameters PART_X and PART_Y, the parameter PART_X represents the number of horizontal average divisions, and the parameter PART_Y represents the number of vertical average divisions.

5. The method for real-time enhancement of low-light-level images based on FPGA according to claim 1, characterized in that: The grayscale histogram of each sub-block of the statistical image includes: Step 4.1: Instantiate RAM1 to count the grayscale histogram of the image sub-block. The read and write addresses of RAM1 correspond to the grayscale value of each pixel in the image sub-block. The data stored in each address is the number of pixels with the corresponding grayscale value. When the rising edge of the field valid signal is valid, RAM1 is cleared. Step 4.2: RAM1 is used as a dual-port RAM. Port A is responsible for writing the number of pixels of the current grayscale level being counted, and port B is responsible for reading the number of pixels his_doutb counted by the current pixel grayscale value. In the process of counting the histogram, the number of each accumulation pix_same_cnt is calculated, and the image data is tapped. If the adjacent grayscale values ​​are different, the number of accumulation pix_same_cnt is 1. When pixels with the same grayscale value appear continuously, it is necessary to continuously add one to count and calculate the continuous number pix_same_cnt. When the adjacent data are different, pix_same_cnt is set to 1. Step 4.3: Add pix_same_cnt to the number of pixels his_doutb of the current pixel grayscale value read out from port B. The result is the number of pixels of the current grayscale value after statistics. At the same time, pull up the write enable signal of port A and write to the corresponding address. When the last line of image data is input, RAM1 completes the grayscale histogram statistics of the image.

6. The method for real-time enhancement of low-light-level images based on FPGA according to claim 1, characterized in that: The grayscale histogram of each image sub-block is cropped and reconstructed, including: Step 5.1: Select a suitable threshold CLIP_LIMIT: Where HEIGHT and WIDTH are the number of rows and columns of the image, PART_X×PART_Y represents the number of sub-blocks, and n is the number of bits of image pixels; Step 5.2: Read the data stored in RAM1 and determine whether the size is greater than the threshold CLIP_LIMIT. If it is, subtract the threshold and add up the results to get the total number of pixels greater than the threshold sum_excess. After reading all the data, divide the number of pixels above the threshold sum_excess by the number of gray levels 2. n , get the average number of pixels that exceed each gray level bin_averate, subtract bin_averate from the threshold CLIP_LIMIT to get the upper limit of the new gray level histogram upper; Step 5.3: Read out the number of pixels of each gray level stored in RAM1 again, and judge the size. If it is greater than the threshold CLIP_LIMIT, assign clip_dina to the threshold CLIP_LIMIT, and the number of pixels of the threshold sum_excess remains unchanged; if it is greater than the threshold CLIP_LIMIT and less than the upper limit upper, assign clip_dina to CLIP_LIMIT, and subtract sum_excess, which is CLIP_LIMIT-his_doutb; if it is less than the upper limit, increase pixel_num by bin_averate. Assign to clip_dina, the number of pixels of the threshold sum_excess minus bin_averate; store clip_dina in RAM2 in gray level order; after completion, right-shift the number of pixels of the threshold sum_excess at this time by the number of bits of pixel data, and add one to prevent it from being 0, and assign the result to step_size; Step 5.4: Next, the data stored in RAM2 is read out in grayscale value order, step_size is increased and assigned to clip_dina, and the number of pixels of the threshold sum_excess is subtracted from step_size, and clip_dina is written into the corresponding address of RAM2. When sum_excess is less than zero, stop reading the data in RAM2, and the redistribution of the grayscale histogram is completed. At this time, the data stored in RAM2 is the histogram of the redistributed image sub-block cropping.

7. The real-time low-light image enhancement method based on FPGA according to claim 1, characterized in that: The grayscale histogram of each image sub-block after cropping and reconstructing is equalized, including: Step 6.1: The cumulative distribution function CDF is defined as the number of pixels less than or equal to a certain gray value. Where h(r i ) is the gray value r i The number of pixels, k is the gray value of the image pixel; The read address of port b of RAM2 is counted from 0 to the maximum gray level, and the data of each address is read out in sequence, and the number of pixels of each gray value read out is accumulated; Step 6.2: Perform a linear transformation on the CDF and map it to the target grayscale value range [0, L-1]; the target grayscale value s k Corresponding to the original gray value r k The mapping relationship is: Where SUM is the total number of pixels in the image sub-block, and [·] indicates a rounding down operation; Multiply the accumulated number of pixels by the maximum grayscale level (2 n -1), divided by the total number of pixels in the image sub-block, the calculated target grayscale value is written into RAM3, the grayscale value of the original pixel is used as the write address, and the converted target grayscale value is used as the storage data.

8. The method for real-time enhancement of low-light-level images based on FPGA according to claim 1, characterized in that: Implement bilinear interpolation between adjacent sub-blocks to eliminate boundary effects between sub-blocks, including: Step 7.1: Determine the interpolation point location: Assume there is an input image, the goal is to calculate the new pixel value based on the pixel point of the image at the coordinate position (x, y); first, we need to find the four neighboring pixel points around the coordinate point, the positions of the four pixel points are Q 11 (x1,y1),Q 12 (x1,y2),Q 21 (x2,y1) and Q 22 (x2,y2), the bilinear interpolation method is as follows: In image processing, the coordinates of (x1, y1) in the formula are set to (0, 0), which can be simplified to: If (x1, y1) is (0, 0), then the grayscale value of each position inside a window of size x2×y2 is calculated; Step 7.2: Perform interpolation calculation: When performing bilinear interpolation calculation, the values ​​of x2 and y2 are related to the sub-block where the pixel is located. The horizontal and vertical coordinates (x, y) inside the window need to be obtained based on the horizontal and vertical coordinates of the pixel in the image sub-block. f(x1, y1), f(x2, y1), f(x1, y2), and f(x2, y2) correspond to the grayscale values ​​after histogram equalization in different sub-blocks. Substituting them into the formula, we can get the final enhanced grayscale value.

Images