A real-time underwater image enhancement method and system based on FPGA

Abstract

The present invention discloses a method and system for real-time underwater image enhancement based on FPGA, the method comprising: S1: clearing the number of each gray level of the stored image, inputting the i-th frame image, respectively calculating and storing the number of each gray level of the frame image and the cumulative number of each gray level; S2: clearing the number of each gray level of the stored i-th frame image, inputting the i+1-th frame image and counting and storing the number of each gray level, after the number of each gray level is counted, sending a statistical completion signal; S3: remapping the image data of the i+1-th frame and the cumulative number of each gray level of the i-th frame image to obtain enhanced image data, and outputting the enhanced image data; S4: starting the cumulative calculation of each gray level of the i+1-th frame image, and saving the cumulative number of each gray level of the i+1-th frame image. The present invention realizes underwater image enhancement processing and improves the real-time performance of image data processing.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to a method and system for real-time underwater image enhancement based on FPGA. Background Art

[0002] With the development of industrialization in my country, the demand for water resources is also growing continuously. However, as an important way to transport water resources, water supply pipelines are subject to severe water leakage. When a pipeline leaks, cracks and holes may appear inside it. This technology is used to enhance the image inside the underwater pipeline, making it easier to detect cracks and holes in the pipeline, thereby achieving the purpose of real-time detection of water leakage.

[0003] The prior art discloses a surveillance video image enhancement technology based on FPGA (Liu Bin. Research on surveillance video image enhancement technology based on FPGA [D]. Changchun University of Technology, 2016.), which mainly includes: 1) calculating the histogram: counting the number of pixels corresponding to each gray level; 2) calculating the new gray value: accumulating the histogram, using the probability distribution function transformation, and obtaining the mapping table; 3) gray mapping: changing the gray value of the original image according to the mapping table to obtain the enhanced image. This scheme implements the histogram equalization algorithm of FPGA, but it only processes single-frame images and cannot perform real-time enhancement processing on multiple frames of images. Summary of the invention

[0004] In order to overcome the defect that the above-mentioned image enhancement method in the prior art cannot realize real-time processing of continuous multiple-frame images, the present invention provides a real-time underwater image enhancement method and system based on FPGA.

[0005] The primary purpose of the present invention is to solve the above technical problems. The technical solution of the present invention is as follows:

[0006] The first aspect of the present invention provides a method for real-time underwater image enhancement based on FPGA, comprising the following steps:

[0007] S1: Clear the number of each gray level of the stored image, input the i-th frame image, calculate and store the number of each gray level of the frame image and the accumulated number of each gray level, where i is a positive integer;

[0008] S2: Clear the number of each gray level of the stored i-th frame image, input the i+1-th frame image and count the number of each gray level and store it. After the number of each gray level is counted, a counting completion signal is issued;

[0009] S3: Remapping the image data of the i+1th frame and the accumulated number data of each gray level of the ith frame image to obtain enhanced image data, and outputting the enhanced image data;

[0010] S4: Start the cumulative calculation of the number of each gray level of the image of the i+1th frame, and save the cumulative number of each gray level of the image of the i+1th frame.

[0011] Furthermore, when the number of each gray level of the stored i-th frame image is cleared in step S2, the number of clock cycles is selected according to the bit width of the input image data. If the data bit width of the image is n, 2 is used. n clock cycles to clear.

[0012] Furthermore, the number of each gray level of each frame image is calculated and stored in a preset gray level statistics table.

[0013] Furthermore, in step S3, the accumulated number data of each gray level used in the remapping calculation is the data of the previous frame image.

[0014] Furthermore, when the accumulation calculation of each gray level number is started, after 256 cycles of accumulation calculation, the accumulation result of each gray level of the i+1th frame image is updated and stored in a preset accumulation statistics table.

[0015] The second aspect of the present invention provides a real-time underwater image enhancement system based on FPGA, comprising: a grayscale statistics module, an accumulation module, a clearing module, and a remapping module, wherein the grayscale statistics module is used to count the number of each grayscale level of the input image;

[0016] The accumulation module is used for accumulating and calculating each gray level of the input image;

[0017] The clearing module is used to clear the grayscale statistics module before the next frame of image is input;

[0018] The remapping module is used to remap the image data of the i+1th frame and the accumulated number data of each gray level of the i-th frame image to obtain enhanced image data.

[0019] Furthermore, the processing steps of the clearing module are:

[0020] Detect the field synchronization signal of the external image acquisition device and pull the wea_clear level high; write data 0 to address 0 of the grayscale statistics table ram_count, and then add one to the address in each cycle until the address count reaches 256, then the address returns to zero; pull the wea_clear level low to stop writing data to the grayscale statistics table ram_count.

[0021] Furthermore, the grayscale statistics module specifically processes the following steps:

[0022] First, the image data is cached into the stack fifo, a data A is read out from the stack fifo, and a register reg is used to cache the data A for one cycle. When the next cycle comes, the second data B in the stack fifo can appear at the same time as the data A in reg.

[0023] Compare the two data of adjacent cycles, i.e., A and B, to see if they are equal. If they are equal, the counter is incremented by 1, and the next data is read into the stack fifo. When the two data of consecutive adjacent cycles are different, the grayscale statistics table ram_count is accessed, and the read value plus the counter value is written into the grayscale statistics table ram_count.

[0024] Furthermore, the specific processing steps of the accumulation module are:

[0025] L1: After detecting the end signal of the grayscale statistics module, initialize sum and the fixed value addra to 0, where sum represents the cumulative sum and addra represents the address;

[0026] L2: The accumulation module starts to read the data data_ in the grayscale statistics table ram_count address, updates the value of sum to data, and writes the value of sum to the address of the accumulation statistics table ram_add.

[0027] L3: The address is incremented by 1, and then the data data_1 in the grayscale statistics table ram_count after the address is incremented by 1 is read, and sum=sum+data_1 is updated, and sum is written into the cumulative statistics table ram_add address after the address is incremented by 1; determine whether the address is greater than 255, if it is less than or equal to 255, repeat step L3, if the address is greater than 255, end the cumulative calculation.

[0028] Furthermore, the processing steps of the remapping module are:

[0029] Get the accumulated number data of each gray level of the previous frame image in the accumulated statistics table ram_add in the accumulation module.

[0030] The first processing result is obtained by multiplying the acquired accumulated number data by the difference between the maximum grayscale and the minimum grayscale of the current frame image;

[0031] The first processing result is divided by the current frame image resolution to obtain enhanced pixels.

[0032] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0033] The present invention can avoid histogram statistical errors caused by grayscale statistics of multiple frames of images by clearing the cache data of the previous frame of image during grayscale statistics. At the same time, histogram accumulation is performed. Based on the correlation between two adjacent frames of images, the accumulated number of each grayscale level of the previous frame is used to perform remapping calculations, thereby ensuring the real-time performance of image data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flow chart of a real-time underwater image enhancement method based on FPGA of the present invention.

[0035] Figure 2 This is a block diagram of a real-time underwater image enhancement system based on FPGA of the present invention.

[0036] Figure 3 This is a processing flow chart of the clearing module according to an embodiment of the present invention.

[0037] Figure 4 This is a processing flow chart of the grayscale statistics module according to an embodiment of the present invention.

[0038] Figure 5 This is a processing flow chart of the accumulation module according to an embodiment of the present invention.

[0039] Figure 6 This is the original underwater image of the interior of the pipeline in the embodiment of the present invention.

[0040] Figure 7 This is an underwater image of the interior of a pipeline after enhancement processing in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0042] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0043] Example 1

[0044] like Figure 1 As shown, the first aspect of the present invention provides a real-time underwater image enhancement method based on FPGA, comprising the following steps:

[0045] S1: Clear the number of each gray level of the stored image, input the i-th frame image, calculate and store the number of each gray level of the frame image and the accumulated number of each gray level, where i is a positive integer;

[0046] S2: Clear the number of each gray level of the stored i-th frame image, input the i+1-th frame image and count the number of each gray level and store it. After the number of each gray level is counted, a counting completion signal is issued;

[0047] S3: Remapping the image data of the i+1th frame and the accumulated number data of each gray level of the ith frame image to obtain enhanced image data, and outputting the enhanced image data;

[0048] S4: Start the cumulative calculation of the number of each gray level of the image of the i+1th frame, and save the cumulative number of each gray level of the image of the i+1th frame.

[0049] It should be noted that when the i-th frame image is first input, the grayscale statistics module counts and stores the number of each grayscale level of the frame. In the present invention, the image uses a preset grayscale statistics table (i.e., ram_count) to store the number of each grayscale level, and uses a preset cumulative statistics table (ram_add) to store the cumulative number of each grayscale level of the first frame.

[0050] Since there is an idle time before each frame of image comes, the number of each gray level in the gray level statistics table of the previous frame is cleared before the next frame of image comes. When clearing the number of each gray level of the stored i-th frame of image in step S2, the number of clock cycles is selected according to the bit width of the input image data. If the data bit width of the image is n, 2 is used. n For example, if the input image data is 8 bits wide, the clear operation requires 2 clock cycles. 8 = 256 clock cycles to clear. After clearing, wait for image data input.

[0051] When a new frame (i.e., the i+1th frame) image begins to be input, the processing operation is completed through two paths. In the first processing path, the number of each gray level of the input frame is counted and stored in a preset gray level statistics table. In the second path, the image data of the input frame is remapped with the data of the first frame image in the cumulative statistics table to obtain the enhanced image data, which is then output.

[0052] It should be noted that when the i+1 frame image remapping calculation is completed, the i+1 frame grayscale statistics calculation is also completed, and the grayscale statistics module sends a completion signal (done signal) to the accumulation module, and the accumulation module reads the i+1 frame data in the grayscale statistics table, and after 256 cycles of accumulation calculation, the accumulation result of each grayscale level of the i+1 frame data is updated and stored in the accumulation statistics table (ram_add in the figure). At this point, the entire algorithm ends.

[0053] It should be noted that, from the perspective of timing, the update calculation of the accumulation module is slower than that of the remapping module, so the data in the accumulation statistics table used by the remapping module is the data of the previous frame image.

[0054] Example 2

[0055] like Figure 2 As shown, the second aspect of the present invention provides a real-time underwater image enhancement system based on FPGA, comprising: a grayscale statistics module, an accumulation module, a clearing module, and a remapping module, wherein the grayscale statistics module is used to count the number of each grayscale level of the input image;

[0056] The accumulation module is used for accumulating and calculating each gray level of the input image;

[0057] The clearing module is used to clear the grayscale statistics module before the next frame of image is input;

[0058] The remapping module is used to remap the image data of the i+1th frame and the accumulated number data of each gray level of the i-th frame image to obtain enhanced image data.

[0059] It should be noted that, in the present invention, before a new frame of image data is input, the statistical data of the number of each gray level of the previous frame of image is cleared by using a clearing module, such as Figure 3 As shown, the specific processing steps are: detect the field synchronization signal of the external image acquisition device (for example, if the ov5640 camera is used, detect the ov5640 field synchronization signal), and pull the wea_clear level high; write data 0 to address 0 of the grayscale statistics table ram_count, and then add one to the address in each cycle until the address count reaches 256, and then the address returns to zero; pull the wea_clear level low to stop writing data to the grayscale statistics table ram_count.

[0060] In the present invention, the grayscale statistics module is used to count the number of grayscale values ​​of each level of the input image data, such as Figure 4 As shown, the specific processing steps are:

[0061] First, the image data is cached into the stack fifo, a data A is read out from the stack fifo, and a register reg is used to cache the data A for one cycle. When the next cycle comes, the second data B in the stack fifo can appear at the same time as the data A in reg.

[0062] Compare the two data of adjacent cycles, i.e., A and B, to see if they are equal. If they are equal, the counter is incremented by 1, and the next data is read into the stack fifo. When the two data of consecutive adjacent cycles are different, the grayscale statistics table ram_count is accessed, and the read value plus the counter value is written into the grayscale statistics table ram_count.

[0063] In the present invention, after the grayscale statistics module counts the number of grayscale values ​​of each level of the input image data, all data from address 0 to 255 are accumulated, and the accumulated results are written into the accumulated statistics table corresponding to the addresses one by one, such as Figure 5 As shown, the specific processing steps are:

[0064] L1: After detecting the end signal of the grayscale statistics module, initialize sum and the fixed value addra to 0, where sum represents the cumulative sum and addra represents the address;

[0065] L2: The accumulation module starts to read the data data in the grayscale statistics table ram_count address, updates the value of sum to data, and writes the value of sum to the address of the accumulation statistics table ram_add.

[0066] L3: The address is incremented by 1, and then the data data_1 in the grayscale statistics table ram_count after the address is incremented by 1 is read, and sum=sum+data_1 is updated, and sum is written into the cumulative statistics table ram_add address after the address is incremented by 1; determine whether the address is greater than 255, if it is less than or equal to 255, repeat step L3, if the address is greater than 255, end the cumulative calculation.

[0067] Example 3

[0068] This embodiment further describes the processing process of the remapping module, and the processing steps of the remapping module are:

[0069] Get the accumulated number data of each gray level of the previous frame image in the accumulated statistics table ram_add in the accumulation module.

[0070] The first processing result is obtained by multiplying the acquired accumulated number data by the difference between the maximum grayscale and the minimum grayscale of the current frame image;

[0071] The first processing result is divided by the current frame image resolution to obtain enhanced pixels.

[0072] like Figure 6 Shown is the original underwater image of the interior of the pipeline. Figure 7 This is the underwater image of the pipeline after enhancement.

[0073] It should be noted that after reading the accumulated number data of each gray level of the previous frame, the remapping module of the present invention first multiplies it by the difference between the maximum gray level and the minimum gray level of the current frame image to obtain a first processing result; then the first processing result is divided by the resolution of the current frame image to obtain the enhanced pixels. This processing process can avoid loss of image data accuracy or data errors.

[0074] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A real-time underwater image enhancement method based on FPGA, characterized in that: The following steps are involved: S1: Clear the number of each gray level of the stored image, input the i-th frame image, calculate and store the number of each gray level of the frame image and the accumulated number of each gray level, where i is a positive integer; S2: Clear the number of each gray level of the stored i-th frame image, input the i+1-th frame image and count the number of each gray level and store it. After the number of each gray level is counted, a counting completion signal is issued; S3: Remapping the image data of the i+1th frame and the accumulated number data of each gray level of the ith frame image to obtain enhanced image data, and outputting the enhanced image data; The accumulated number data of each gray level used in the remapping calculation is the data of the previous frame image; S4: Start the cumulative calculation of the number of each gray level of the image of the i+1th frame, and save the cumulative number of each gray level of the image of the i+1th frame.

2. The method for real-time underwater image enhancement based on FPGA according to claim 1, characterized in that: When clearing the number of each gray level of the stored i-th frame image in step S2, the number of clock cycles is selected according to the bit width of the input image data. If the data bit width of the image is n, 2 is used. n clock cycles to clear.

3. The method for real-time underwater image enhancement based on FPGA according to claim 1, characterized in that: The number of each gray level in each frame of image is calculated and stored in a preset gray level statistics table.

4. The method for real-time underwater image enhancement based on FPGA according to claim 1, characterized in that: When the accumulation calculation for each gray level is started, after 256 cycles of accumulation calculation, the accumulation result of each gray level of the i+1th frame image is updated and stored in a preset accumulation statistics table.

5. A real-time underwater image enhancement system based on FPGA, characterized in that: include: Grayscale statistics module, accumulation module, clearing module, remapping module, wherein the grayscale statistics module is used to count the number of each grayscale level of the input image; the specific processing steps of the grayscale statistics module are: First, the image data is cached into the stack fifo, a data A is read out from the stack fifo, and a register reg is used to cache the data A for one cycle. When the next cycle comes, the second data B in the stack fifo can appear at the same time as the data A in reg. Compare the two data of adjacent cycles, i.e., A and B, to see if they are equal. If they are equal, the counter is incremented by 1, and the next data is read into the stack fifo. When the two data of consecutive adjacent cycles are different, the grayscale statistics table ram_count is accessed, and the read value plus the counter value is written into the grayscale statistics table ram_count. The accumulation module is used for accumulating and calculating each gray level of the input image; The clearing module is used to clear the grayscale statistics module before the next frame of image is input; The remapping module is used to remap the image data of the i+1th frame and the accumulated number data of each gray level of the i-th frame image to obtain enhanced image data.

6. The FPGA-based real-time underwater image enhancement system according to claim 5, characterized in that: The processing steps of the clearing module are: Detect the field synchronization signal of the external image acquisition device and pull the wea_clear level high; write data 0 to address 0 of the grayscale statistics table ram_count, and then add one to the address in each cycle until the address count reaches 256, then the address returns to zero; pull the wea_clear level low to stop writing data to the grayscale statistics table ram_count.

7. The FPGA-based real-time underwater image enhancement system according to claim 5, characterized in that: The specific processing steps of the accumulation module are: L1: After detecting the end signal of the grayscale statistics module, initialize sum and the fixed value addra to 0, where sum represents the cumulative sum and addra represents the address; L2: The accumulation module starts to read the data data in the grayscale statistics table ram_count address, updates the value of sum to data, and writes the value of sum to the address of the accumulation statistics table ram_add; L3: The address is incremented by 1, and then the data data_1 in the grayscale statistics table ram_count after the address is incremented by 1 is read, and sum=sum+data_1 is updated, and sum is written into the cumulative statistics table ram_add address after the address is incremented by 1; determine whether the address is greater than 255, if it is less than or equal to 255, repeat step L3, if the address is greater than 255, end the cumulative calculation.

8. The FPGA-based real-time underwater image enhancement system according to claim 5, characterized in that: The processing steps of the remapping module are: Get the accumulated number data of each gray level of the previous frame image in the accumulated statistics table ram_add in the accumulation module. The first processing result is obtained by multiplying the acquired accumulated number data by the difference between the maximum grayscale and the minimum grayscale of the current frame image; The first processing result is divided by the current frame image resolution to obtain enhanced pixels.

Images