FPGA-based low-illumination video enhancement hardware implementation method

Abstract

The application discloses a low-illumination video enhancement hardware implementation method based on FPGA, which comprises the following steps: RGB and HSV space mutual conversion; for V channel, the pixel number and total gray value of input image are obtained; a brightness evaluation module is used for quantitatively evaluating and grading the low-illumination condition of the image according to the average brightness value of the image, and a damping coefficient is added to prevent parameter mutation; a suitable amplitude limiting value is determined according to the evaluation of the low-illumination, and the histogram information of an image is counted and stored; the statistical result is normalized to obtain a mapping table; a bilinear interpolation is used to eliminate the blocking effect of the block histogram equalization; the motion intensity of the pixel point is judged to distinguish the foreground and the background; a bilateral filtering algorithm is used for spatial denoising of the background; and the foreground is temporally denoised. The CLAHE algorithm is used to keep the color distortion-free, improve the brightness and contrast of the low-illumination image, and enhance the degree of change according to the scene in real time, and then the video 3D denoising algorithm is used to improve the imaging quality.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically a hardware implementation method for low-light video enhancement based on FPGA. Background Technology

[0002] FPGAs, due to their parallel computing and programmability, are increasingly used in video and image processing. In low-light environments, images captured at night suffer from low brightness, reduced contrast, and significant noise due to the unique characteristics of nighttime illumination. The video stream from the image sensor is inevitably affected by noise during acquisition, processing, and transmission. Uneven illumination distribution also exists, with only localized areas receiving sufficient light while other areas suffer from reduced detail visibility and more complex noise distribution. Furthermore, non-white light illumination often leads to color deviations in objects at night. Histogram equalization is a widely used method for enhancing image contrast, but it suffers from localized overexposure and poor enhancement in some areas. The improved CLAHE algorithm significantly improves this, but it is complex, requires numerous iterations, is difficult to process in real-time, and cannot automatically adjust the enhancement intensity. The mainstream video denoising method is currently the 3D video denoising method, which combines spatial domain denoising and temporal domain denoising. However, spatial domain denoising can destroy image details, and the choice of method has a significant impact on the denoising effect. Temporal domain denoising ignores spatial correlation, is affected by moving objects, and has high complexity and high resource consumption, making it unsuitable for implementation on FPGA platforms. Summary of the Invention

[0003] The purpose of this invention is to provide a hardware implementation method for low-light image enhancement based on FPGA, which can be ported into a camera. It can adaptively enhance video contrast and brightness according to the environmental scene, improve visual effects, maintain color accuracy, and reduce the impact of noise. The algorithms used are all parallelized, which improves the algorithm speed.

[0004] The technical solution to achieve the objective of this invention is: a hardware implementation method for low-light video enhancement based on FPGA, the specific steps of which are as follows:

[0005] Step 1: Construct an RGB to HSV color space module inside the FPGA to obtain the V channel image brightness information;

[0006] Step 2: Statistically analyze the V channel information, calculate the average brightness of the frame, process the average brightness with a damping coefficient, and determine whether it is a low-light image based on the processing result. If it is, calculate the limiting value and proceed to the next step; otherwise, directly output the original image.

[0007] Step 3: Construct a block histogram statistics module inside the FPGA and use a logic design that reduces RAM read / write operations to obtain histogram information for each block of the image;

[0008] Step 4: Construct a normalization calculation and interpolation module inside the FPGA to perform histogram equalization on each block of the image and bilinear interpolation mapping between blocks;

[0009] Step 5: Build a filtering module inside the FPGA to perform bilateral filtering and noise reduction on the image;

[0010] Step 6: Construct a motion estimation module inside the FPGA, differ between the previous frame and the current frame, filter out salt-and-pepper noise through median filtering, binarize the result using the average brightness after damping coefficient processing and threshold to separate the foreground and background, add morphological filtering to construct a mask for the background and foreground, and perform temporal filtering on pixels that are not identified as motion points.

[0011] Preferably, the method for implementing color space conversion by the RGB to HSV color space module is as follows:

[0012] The data for the H, S, and V channels are obtained based on the formula:

[0013] max = max(R,G,B);

[0014] min = min(R,G,B);

[0015]

[0016] V = max(R,G,B);

[0017]

[0018] The data for the H, S, and V channels are obtained, where R, G, and B are the grayscale values ​​of the R, G, and B color channels, respectively.

[0019] Preferably, the specific method for statistically analyzing the V channel information to obtain the overall brightness of the input image is as follows:

[0020] When image data is written to and pixel values ​​are read, two 29-bit wide registers are set to store the number of input pixels and the sum at the current moment. The registers are updated every cycle. After one frame of data, the cumulative number of pixels and the sum of V channel data for this image frame are obtained. The average value is obtained by using a divider, which is the average brightness of the current frame.

[0021] Preferably, the specific formula for processing the damping coefficient of the brightness information is as follows:

[0022] g(n)=(1-s)*g(n-1)+s*g

[0023] Where g is the value to be converged, i.e., the average value, n is the current frame number, g(n-1) is the value after processing the previous frame, g(n) is the value after processing the current frame, and s is the damping coefficient.

[0024] Preferably, the image is determined to be a low-light image based on the processing result. If so, the specific method for calculating the limiting value is as follows:

[0025] When the calculated g(n) is less than or equal to the set threshold, it is determined to be a low-light image, and a limiting value is calculated, specifically:

[0026] The amplitude limit is determined based on the magnitude of the average value, specifically as follows:

[0027]

[0028] Where up is the amplitude limit value and y is the brightness information, which is g(n) here.

[0029] Preferably, the specific method for obtaining the histogram information of each block of the image is as follows:

[0030] Instantiate a RAM to store the statistical results. The data bit width is 12 bits and the data depth is 16384, which is 14 address lines. The high six bits are row addressing and column addressing, and the low eight bits are pixel data.

[0031] The image is divided into 6 rows and 8 columns, and each region has an independent address;

[0032] Set a write enable and a counter. When the data written at a certain moment is the same as the data read at the previous moment, the write enable is pulled low and the counter is incremented by 1. The same pixel data is counted on the counter and finally written to RAM. For pixels with a value greater than the up setting value, the up value is used instead of the value written to RAM and the number of such pixels is stored.

[0033] The setup and transition process is as follows: An IDLE reset state is set. Upon receiving the falling edge signal of the field signal, the histogram statistics state is entered. Upon receiving the rising edge signal of the field signal, it indicates that the effective pixel input is complete. The state then enters the cropping value calculation state, distributing the total number of values ​​exceeding the amplitude evenly across each pixel's grayscale level. Excess values ​​are preferentially distributed to lower pixel levels, i.e., using a divider to calculate the quotient and remainder. Based on the indication signal, the RAM2 clearing module is entered. RAM2 is the memory that stores the cumulative distribution histogram. Each address bit in RAM2 is written to 0, and a counter is used to calculate the time. After clearing is complete, the accumulation state is entered. In the accumulation state, the data preceding each grayscale level is added together, along with the quotient and remainder from the previous cropping, to obtain the cumulative distribution histogram with contrast limitation, which is stored in RAM2. Similarly, a counter is used to calculate the time. After completion, RAM1 (the RAM for histogram statistics) is cleared. After clearing is complete, the system enters a waiting state, awaiting the arrival of the next frame of image data.

[0034] Preferably, the formula for bilinear interpolation is:

[0035]

[0036] x, y, r, and s represent the distances from a pixel in the image patch to the boundaries of its left, right, top, and bottom sub-blocks, respectively. (i-1,j-1) map (i,j-1) map (i-1,j) map (i,j) P represents the mapping function for the four neighboring image blocks of the sub-block containing the pixel. out These are the interpolated pixel values.

[0037] Preferably, a filtering module is built inside the FPGA. The specific method for bilateral filtering and denoising the image is as follows: the 3*3 value domain and spatial domain filtering kernels are calculated in MATLAB, the data is initialized in the ROM of Vivado, and when the data enters, the data is directly read from the ROM to obtain the bilateral filtering convolution kernel, and the result is obtained after the operation.

[0038] Preferably, the specific method for median filtering to remove salt-and-pepper noise is as follows: instantiate four FIFOs with a depth of one row of data as row buffers. The four rows of buffered data and the data read and written together form a 5*5 convolution kernel. The 25 data are grouped and sorted to obtain the median value instead of the value at the center of the convolution kernel.

[0039] Preferably, the result after median filtering is binarized to separate the foreground and background, and morphological filtering is added to construct a mask for the background and foreground. The specific method for temporal filtering of pixels not identified as moving points is as follows:

[0040] For the processed frame difference image, the mean of the 3*3 neighborhood of each pixel is calculated as the motion judgment parameter P. The pixel noise intensity is calculated for the current frame using the following formula:

[0041]

[0042] Where N is the noise intensity, w(i,j) is the pixel data in the i-th row and j-th column of the 3*3 neighborhood, and avar is the mean. P and N are compared. If P is greater than N, the pixel is determined to be a moving pixel and its gray value is set to 255. Otherwise, it is set to 0, thus obtaining the binarized image.

[0043] The binarized image is processed by morphological filtering with a closing operation to obtain a mask image of the moving points;

[0044] Based on the mask image information, moving points are left unprocessed, while non-moving points undergo temporal filtering. The specific formula for temporal filtering is as follows:

[0045]

[0046] Where g(x,y) is the processing result, and f(x,y) is the current frame. This is the reference frame.

[0047] Preferably, the specific method for obtaining the mask image of the moving point after morphological filtering of the binarized image by closing operation is as follows: first dilate the binary image and then erode it. The dilation method is that if there is a point with a gray value of 255 in the 3*3 neighborhood of the pixel, then the gray value of the entire neighborhood is set to 255. The erosion method is that if there is a point with a gray value of 0 in the 3*3 neighborhood of the pixel, then the gray value of the entire neighborhood is set to 0.

[0048] Compared with existing technologies, the present invention has the following significant advantages: (1) Enhancement processing of color images in the HSV color space maintains image saturation and prevents color distortion. (2) By statistically analyzing image information and adjusting the contrast limit in real time, it has a good effect on images in different illumination scenarios. (3) A dynamic threshold method is used for motion judgment, with different thresholds for each pixel. Median filtering and morphological filtering are added to the mask image for motion point judgment, which improves the accuracy of region division. (4) The selected bilateral filtering spatial filtering and temporal filtering methods have good noise reduction effects and also preserve edge and detail information well. (5) The algorithm is implemented on FPGA, which has good versatility and improves real-time performance.

[0049] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0050] Figure 1 This is the overall structural framework of the present invention.

[0051] Figure 2 It is the V channel diagram of the HSV color space.

[0052] Figure 3 This is a graph of histogram equalization before interpolation.

[0053] Figure 4 This is a graph showing the elimination of block artifacts after interpolation.

[0054] Figure 5 This is a diagram showing the specific logical structure of video 3D denoising.

[0055] Figure 6 This is the image before denoising after enhancement.

[0056] Figure 7 It is the image after enhancement and denoising. Detailed Implementation

[0057] like Figure 1 The diagram shown is the overall block diagram of the low-light image enhancement hardware implementation method of the present invention, which is divided into two main modules: enhancement and denoising.

[0058] like Figure 2 As shown, the V channel of the HSV color channel reflects the brightness information of the image, and enhancing it will not affect the original color and color saturation information of the image.

[0059] like Figure 3 As shown, after the block histogram processing, each block of the image is enhanced to different degrees. The enhancement effects of brighter and darker areas are different, but obvious block effects are produced. Figure 4 This is the effect after interpolation elimination.

[0060] like Figure 5 As shown, the structure of the video 3D denoising data stream is divided into obtaining the frame difference map, frame difference map preprocessing, and spatial and temporal filtering.

[0061] A hardware implementation method for low-light image enhancement based on FPGA is proposed, targeting solutions for improving imaging performance of small intelligent cameras under low-light conditions. This method ports related software-serial algorithms to hardware parallel computing, effectively leveraging the speed and resource advantages of FPGA. Furthermore, the algorithm is improved to allow the camera imaging to adaptively change relevant parameters and effects according to the environment. The noise reduction part is reasonably set, and different thresholds are applied based on the motion of each pixel to accurately distinguish between moving and non-moving points. Various preprocessing methods are incorporated to obtain a better motion region mask image. The specific steps are as follows:

[0062] Step 1: Construct an RGB-to-HSV color space module within the FPGA to obtain the V channel image brightness information. The color space conversion is implemented using the following formula:

[0063] max = max(R,G,B);

[0064] min = min(R,G,B);

[0065]

[0066] V = max(R,G,B);

[0067]

[0068] The system obtains data from three channels: H, S, and V. R, G, and B represent the grayscale values ​​of the R, G, and B color channels, respectively. The V channel is input to the next module for further processing. The H and S channels are stored in registers and timed until the V channel processing is complete, after which the data is converted back to RGB color space.

[0069] Step 2: Calculate the V channel information to obtain the overall brightness of the input image. When image data is written and pixel values ​​are read one by one, two 29-bit registers are set to store the current number of input pixels and their cumulative sum. This is updated every cycle. After one frame of data, the cumulative number of pixels and the cumulative sum of the V channel data for that image frame are obtained. A divider is used to obtain the average value, which is the average brightness of the current frame. A damping factor is then used to prevent abrupt changes. The damping factor formula is:

[0070] g(n)=(1-s)*g(n-1)+s*g

[0071] Where g is the value to be converged, n is the current frame number, g(n-1) is the value after processing in the previous frame, and g(n) is the value after processing in the current frame. s is the damping coefficient, which can be set to 0.2. 1-s to the power of 30 is 0.001, which is negligible, so it can be said that g(n) converges to g after 30 processing iterations. Substituting the average value into g, g(n) converges to the average after 30 frames. Method for determining if it is a low-light image: When g(n) is greater than 120, it is determined to be a normal lighting image, and the original image is output without processing by the method in this paper; otherwise, continue with all the following steps. The limiting value of the histogram statistics module is determined based on the magnitude of g(n). Specifically:

[0072]

[0073] Where up represents the amplitude limit value.

[0074] Step 3: Construct a block histogram statistics module inside the FPGA to obtain histogram information for each block of the image. The specific method for histogram statistics is as follows: Instantiate a RAM as the storage for statistical results, with a data bit width of 12 bits and a data depth of 16384, i.e., 14 address lines. The high six bits are for row and column addressing, and the low eight bits are for pixel data. The image is divided into 6 rows and 8 columns, and each block has an independent address. The conventional storage method is to increment the data at the corresponding address when image data arrives, and each pixel requires a data read and write operation. However, under low illumination, many pixel data intervals are concentrated in a very small range, inevitably resulting in a large number of pixel data repetitions. Therefore, a write enable and counter are set. When the data written at a certain moment is the same as the data read at the previous moment, the write enable is pulled low and the counter is incremented by 1. In this way, the same pixel data will be counted on the counter and finally written to RAM together, reducing the number of RAM read and write operations and reducing the possibility of errors. Furthermore, for pixels with a value greater than the up setting, the up value is used instead of being written to RAM, and the number of such pixels is stored. The settings and transitions are as follows: An IDLE reset state is set. Upon receiving the falling edge of the field signal, the histogram statistics state is entered. Upon receiving the rising edge of the field signal, indicating that all valid pixels have been input, the state for calculating the cropping value is entered. The total number of pixels exceeding the amplitude is evenly distributed across each pixel's grayscale level, with excess pixels preferentially distributed to lower pixel levels, i.e., using a divider to calculate the quotient and remainder. After division, the RAM2 clearing module is entered according to the indicator signal. RAM2 is the memory that stores the cumulative distribution histogram. Each address bit in RAM2 is written to 0, and a counter is used to calculate the time. After clearing, the system enters the accumulation state. In the accumulation state, the data preceding each grayscale level is added together, along with the quotient and remainder from the previous cropping, to obtain the cumulative distribution histogram with contrast limitation, which is stored in RAM2. The counter is used to calculate the time again. After completion, RAM1 (the RAM for histogram statistics) is cleared. After clearing, the system enters a waiting state, awaiting the arrival of the next frame of image data.

[0075] Step 4: A normalization calculation and interpolation module is built inside the FPGA to perform histogram equalization on each block of the image, and bilinear interpolation mapping between blocks. After reading the cumulative distribution histogram data, dividing it by the total number of pixels in each block yields the grayscale transformation mapping table. However, because the image is equalized in blocks, the result will produce obvious block artifacts, such as… Figure 3 As shown. The solution is to implement bilinear interpolation, where the mapping value of each pixel in an image block is obtained by interpolating the mapping values ​​of its neighboring image blocks. The specific interpolation formula is:

[0076]

[0077] x, y, r, and s represent the distances from a pixel in the image patch to the boundaries of its left, right, top, and bottom sub-blocks, respectively.(i-1,j-1) map (i,j-1) map (i-1,j) map (i,j) P represents the mapping function for the four neighboring image blocks of the sub-block containing the pixel. out This represents the interpolated pixel values. Since the mapping values ​​of four image blocks need to be used simultaneously, the cumulative distribution histogram results need to be stored in four RAMs. Address logic is written to allow the interpolation module to call these blocks simultaneously. The parameters i, j, s, and r are determined based on the pixel-by-pixel and row-by-row characteristics of the input data and the different regions they occupy. The coefficient i can be set to 240, decreasing with each image input cycle except for the left boundary and bounded sub-blocks. The left boundary is always 240, and the bounded sub-blocks are always 0. Similarly, the coefficient s initially has a value of 180, decreasing at the upper and lower boundaries. The upper boundary region remains at 180, and the lower boundary is 0. Region identification also uses a counting method to set a flag signal to identify the region type. After calculating the values ​​of j and r, and synchronizing i and s, interpolation is performed within four cycles. The original mapping values ​​are used for corner blocks, and one-dimensional linear interpolation is performed between adjacent image blocks for image edges. The interpolated result is... Figure 4 The image shown demonstrates that the block effect has disappeared, the transitions between image blocks are smooth, and the visual effect is good.

[0078] Step 5: Construct a filtering module within the FPGA to perform bilateral filtering for image denoising. Implement a simplified bilateral filter on the FPGA. Pre-calculate the 3x3 range and spatial domain filter kernel in MATLAB. Experimentally increase the upper and lower limits of the data, and perform fixed-point processing by a factor of 1024. Initialize the kernel in Vivado's ROM. When data enters, directly read the data from the ROM to obtain the bilateral filter convolution kernel, and then perform the calculation to obtain the result.

[0079] Step 6: Construct a motion estimation module within the FPGA. Difference the previous frame with the current frame, filter out salt-and-pepper noise using median filtering, binarize the result with an appropriate threshold to separate the foreground and background, and add morphological filtering to construct a mask for the background and foreground. Pixels not identified as motion points undergo temporal filtering. This invention uses DDR3 for video frame buffering, instantiates the Vivado MIG 7 IP core, and writes read / write control logic to implement multi-channel buffer data read / write. When a video frame arrives, the video data from the previous frame is synchronously read and differentially analyzed to obtain the data differences between adjacent frames. To avoid the influence of impulse noise on the difference results, a 5x5 window size median filtering is used. This is achieved by instantiating four FIFOs with a depth of one row as row buffers. The four rows of buffered data and the incoming data together form a 5x5 convolution kernel. The 25 data points are grouped and sorted, and the median value is used to replace the value at the center of the convolution kernel. For the processed frame difference image, the mean of the 3x3 neighborhood of each pixel is calculated as the motion judgment parameter P. The selection of the neighborhood is similar to median filtering, using two FIFO buffers to buffer two rows of data to obtain a 3x3 window. Simultaneously, the pixel noise intensity of the current frame is calculated. Specifically, the average value of the 3x3 neighborhood is calculated, and the difference between each data point in the 3x3 neighborhood and the mean is calculated and summed. Finally, the average is taken. The expression is:

[0080]

[0081] Where N is the noise intensity, w(i,j) is the pixel data in the i-th row and j-th column of a 3*3 neighborhood, and avar is the mean. P and N are compared; if P is greater than N, the pixel is identified as a moving pixel and its grayscale value is set to 255; otherwise, it is set to 0, thus obtaining a binarized image. The binarized image is then processed by morphological filtering, such as closing operations, to obtain a mask image for the moving points. Specifically, this involves first dilation and then erosion. The dilation method sets the grayscale value of the entire neighborhood to 255 if there is a pixel with a grayscale value of 255 in the 3*3 neighborhood. The erosion method sets the grayscale value of the entire neighborhood to 0 if there is a pixel with a grayscale value of 0 in the 3*3 neighborhood. Based on the mask image information, moving points are not processed; only delayed snapshots are performed. Non-moving points undergo temporal filtering, and finally, these are combined to form the 3D video denoising result. The specific formula for temporal filtering is:

[0082]

[0083] Where g(x,y) is the processing result, and f(x,y) is the current frame. Use the reference frame. The final processing result is as follows: Figure 7 You can see that Figure 6 Compared to other methods, the noise reduction effect is obvious, and the details are well preserved.

Claims

1. A low-light video enhancement hardware implementation method based on FPGA, characterized in that, The specific steps are as follows: Step 1: Construct an RGB-to-HSV color space module inside the FPGA to obtain the V-channel image brightness information; Step 2: Count the V-channel information, calculate the average brightness, and process the average brightness with a damping coefficient. If the result indicates that it is a low-illumination image, calculate the clipping value and proceed to the next step. If not, directly output the original image; Step 3: Construct a block histogram statistics module inside the FPGA and use a logic design to reduce ram read / write to obtain histogram information for each block of the image; Step 4: Construct a normalization calculation and interpolation module inside the FPGA to perform histogram equalization on each block of the image and bilinear interpolation mapping between blocks; Step 5: Construct a filtering module inside the FPGA to perform bilateral filtering to denoise the image; Step 6: Construct a motion estimation module inside the FPGA to difference the current frame from the previous frame, filter out salt and pepper noise through median filtering, binarize the result using the average brightness processed with a damping coefficient and a threshold to separate the foreground and background, and add a morphological filtering process to construct a mask for the background and foreground. For pixels not determined as motion points, perform temporal filtering as follows: For the processed frame difference image, calculate the average of the 3*3 neighborhood of each pixel point as the motion judgment parameter P. Calculate the pixel noise intensity for the current frame as follows: where N is the noise intensity, w(i,j) is the pixel data in the 3*3 field, avr is the average; compare P and N. If P is greater than N, the pixel is determined as a motion pixel point, and the pixel gray value is set to 255. Otherwise, it is set to 0, and a binarized image is obtained; After morphological filtering of the binarized image through closing operation, a mask image of motion points is obtained; According to the mask image information, do not process the motion points, and perform temporal filtering on the non-motion points. The specific formula for temporal filtering is as follows: where g(x, y) is the processing result, f(x, y) is the current frame, is the reference frame.

2. The FPGA-based low-light video enhancement hardware implementation method according to claim 1, wherein, The implementation method of the RGB-to-HSV color space module for color space conversion is as follows: According to the formula, obtain H, S, and V channel data: max = max(R, G, B); min = min(R, G, B); V = max(R, G, B); Obtain H, S, and V channel data, where R, G, and B are the gray values of the R, G, and B color channels, respectively. 3.The FPGA-based low-light video enhancement hardware implementation method of claim 1, wherein, The specific method for counting V-channel information to obtain the overall brightness of the input image is as follows: When writing image data to read pixel values, set two 29-bit registers to save the number of input pixels at the current time and the cumulative sum. Update every period, and after a frame of data, obtain the pixel count and V-channel data cumulative sum for this image frame. Use a divider to obtain the average value, which is the average brightness of the current frame.

4. The FPGA-based low-light video enhancement hardware implementation method of claim 3, wherein, The specific formula for damping coefficient processing of brightness information is as follows: g(n) = (1-s)*g(n-1) + s*g where g is the value to be converged, i.e., the average value, n is the current frame number, g(n-1) is the processed value of the previous frame, g(n) is the processed value of the current frame, and s is the damping coefficient.

5. The FPGA-based low-light video enhancement hardware implementation method of claim 4, wherein, According to the processing result, it is judged whether it is a low-illumination image. If yes, the specific method for calculating the clipping value is as follows: When the obtained g(n) is less than or equal to the set threshold, it is determined that it is a low-illumination image, and the clipping value is obtained, which is specifically as follows: The clipping value is determined according to the size of the average value, which is specifically as follows: Wherein, up is the size of the clipping value, and y is the luminance information, which is g(n) here.

6. The FPGA-based low-light video enhancement hardware implementation method according to claim 1, wherein, The specific method for obtaining the histogram information of each block of the image is as follows: An example of a RAM is used to save the statistical results, the data bit width is 12 bits, the data depth is 16384, that is, 14 address lines, the upper six bits are row selection, the lower eight bits are pixel data; The image is divided into 6 rows and 8 columns, and each block has independent addresses; A write enable and a counter are set. When the data written at a moment is the same as the data read at the last moment, the write enable is pulled down, and the counter is incremented by 1. The same pixel data is counted on the counter, and finally written into the RAM. For the pixel whose value is greater than up, the value up is used to replace the value written into the RAM, and the number of such pixels is stored. The state and transition condition are set as follows: an IDLE reset state is set, the falling edge signal of the field signal is obtained, the histogram counting state is entered, the rising edge signal of the field signal is obtained, the input of the effective pixel is completed, the clipping value calculation state is entered, the total number of pixels exceeding the amplitude is evenly distributed to each pixel gray level, and the excess part is preferentially distributed to the low pixel level, that is, the quotient and the remainder are calculated by using the divider; according to the indication signal, the RAM2 zero clearing module is entered, and the RAM2 is a memory for saving the cumulative distribution histogram. Each bit address of the RAM2 is written with 0, and the time is counted by using the counter. After the zero clearing is completed, the accumulation state is entered. In the accumulation state, the data before each gray level is added, and the quotient and the remainder obtained by clipping are added, so that the cumulative distribution histogram of the limited contrast is obtained and stored in the RAM2. The time is counted by using the counter, and after the completion, the RAM1 (the histogram counting RAM) is cleared, and after the zero clearing is completed, the waiting state is entered, and the next frame of image data is waited.

7. The FPGA-based low-light video enhancement hardware implementation method of claim 1, wherein, The formula of the bilinear interpolation is as follows: x, y, r, s are the distances from the pixel to the left, right, top and bottom partitioning sub-block boundaries, map (i-1,j-1) , map (i,j-1) , map (i-1,j) , map (i,j) are the mapping functions of the four neighboring image blocks of the sub-block in which the pixel is located, P out is the interpolated pixel value.

8. The FPGA-based low-light video enhancement hardware implementation method of claim 1, wherein, The specific method for performing the bilateral filtering denoising on the image by constructing the filtering module in the FPGA is as follows: the 3*3 value domain and spatial domain filtering kernel is calculated on the matlab, the data is initialized in the vivado ROM, the data is directly read out from the ROM to obtain the bilateral filtering convolution kernel when the data is entered, and the result is obtained after operation.

9. The FPGA-based low-light video enhancement hardware implementation method of claim 1, wherein, The specific method for filtering the salt and pepper noise by median filtering is as follows: four FIFOs with a row of data size are instantiated as row buffers, four row buffer data and the data read and written together form a 5*5 size convolution kernel, 25 data groups are sorted to obtain the median value instead of the center value of the convolution kernel.

Images