FPGA-based non-cooled infrared image enhancement system and method

Abstract

The application belongs to the technical field of infrared image enhancement methods, and particularly relates to a non-cooled infrared image enhancement system and method based on FPGA, which takes FPGA as a processing core, combines a bilateral filtering algorithm and a gamma correction algorithm, and performs real-time enhancement on an infrared image. The bilateral filtering algorithm simultaneously considers distance between pixel points and gray value difference in the processing process, can clearly retain original details of the image while enhancing the image effect, and achieves the purpose of edge preservation and noise removal. On this basis, the gamma correction algorithm is used to make the denoised image have stronger contrast, and effectively improve the image quality. Meanwhile, in the system framework, the chip SRAM-A and the chip SRAM-B are multiplexed, so that the whole system can utilize less storage resources to realize fast processing of larger data, thereby meeting the real-time requirement of the image processing system.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of infrared image enhancement methods, and particularly relates to a non-cooled infrared image enhancement system and method based on FPGA. BACKGROUND

[0002] Non-cooled infrared detectors have been widely used in aerospace, medicine, security and industrial production technical fields. With the continuous increase of application range, people's requirements for infrared images are also getting higher and higher. However, in the process of working, the infrared detector will be affected by environmental temperature, circuit noise and detector working temperature fluctuation, etc., so that the infrared image contrast is low and the details are fuzzy. In order to obtain higher quality infrared images, people begin to try to build image processing systems, and use image processing algorithms to optimize the original infrared image and enhance the contrast of the infrared image and highlight the image details.

[0003] At present, the commonly used image enhancement algorithms in the constructed image processing system include histogram equalization, median filtering, Laplace operator and Gaussian filtering methods. Among these methods, histogram equalization, median filtering and Laplace operator are relatively simple in algorithm and easy to implement, but such methods cannot well preserve the details in the image while enhancing the image. Compared with the first three algorithms, the Gaussian filtering has better enhancement effect, but the calculation amount is large, so that the running speed of the whole infrared image processing system cannot meet the application requirements, and a large storage resource is also needed to cache the data generated in the processing process. SUMMARY

[0004] The application aims to provide a non-cooled infrared image enhancement system based on FPGA, so as to realize the enhancement of the image while clearly preserving the image details, achieve the purpose of edge-preserving enhancement, reduce the calculation amount and improve the operation efficiency.

[0005] To solve the above technical problems, the application adopts the following technical scheme:

[0006] A non-cooled infrared image enhancement system and method based on FPGA, comprising an image acquisition control module, a non-cooled infrared detection core, a main control module, a chip SRAM-A, a chip SRAM-B, a bilateral filtering processing module, a gamma correction processing module, an image output module, a digital-analog conversion chip and a VGA output interface.

[0007] The non-cooled infrared detection core is connected with the image acquisition control module, and is used for completing the acquisition of image data.

[0008] The image acquisition control module is interactively connected with the main control module, and is configured to configure a working mode of the uncooled infrared detection core according to requirements; and receive and send image data collected by the uncooled infrared detection core;

[0009] The main control module has a first end interactively connected with the image acquisition control module, a second end interactively connected with the chip SRAM-A, a third end interactively connected with the bilateral filter processing module, a fourth end interactively connected with the chip SRAM-B, a fifth end interactively connected with the gamma correction processing module, and a sixth end connected with the image output module; the main control module is configured to control the running of the bilateral filter processing module and the gamma correction processing module, and control the reading and writing of the image data in the chip SRAM-A and the chip SRAM-B;

[0010] The chip SRAM-A is configured to cache the image data collected by the uncooled infrared detection core; and the chip SRAM-B is configured to cache the image data processed by the bilateral filter.

[0011] The bilateral filter processing module is configured to acquire the image data provided by the image acquisition control module, and perform bilateral filter processing on the image data.

[0012] The gamma correction processing module is configured to perform gamma correction processing on the image data processed by the bilateral filter.

[0013] The image output module is connected with the VGA output interface through a digital-to-analog conversion chip, and is configured to receive the image data processed by the gamma correction, control the output timing sequence of the image data, and make the image data adapt to the timing sequence requirement of the output mode; the image data output by the image output module is processed by the digital-to-analog conversion chip, and then output through the VGA interface.

[0014] Further, the bilateral filter processing module is internally provided with four FIFO cache units and a random access memory (RAM); the four FIFO cache units are configured to cache the image data in the chip SRAM-A acquired through the main control module; the random access memory (RAM) is pre-stored with a brightness proximity weight lookup table taking a brightness difference value as an address and a gamma function lookup table taking a pixel point gray value as an address; the brightness proximity weight lookup table contains an address and a brightness proximity weight; and the gamma function lookup table contains a pixel point gray value and a gamma function value.

[0015] An uncooled infrared image enhancement system based on an FPGA, and a method for image enhancement of the system includes the following steps:

[0016] S1, an image acquisition control module configures a working mode of an uncooled infrared detection core according to requirements, and the configured working mode includes image acquisition mode, pixel point precision and other information;

[0017] S2, when the configuration is completed, the non-cooled infrared detection core starts to collect image data, and sends the collected image data to the main control module through the image collection control module, and the main control module controls writing of the image data into the chip SRAM-A for buffering;

[0018] S3, when the image data is written, the main control module sends a start signal to the bilateral filter processing module; after receiving the start signal, the bilateral filter processing module acquires the image data in the chip SRAM-A through the main control module, and then processes the image data by using a bilateral filter algorithm;

[0019] S4, when the bilateral filter processing is completed, the bilateral filter processing module transmits the image data processed by the bilateral filter to the main control module, and sends a completion signal; after receiving the completion signal, the main control module controls the image data processed by the bilateral filter to be buffered into the chip SRAM-B, and sends a start signal to the gamma correction processing module; after receiving the start signal, the gamma correction processing module reads the image data processed by the bilateral filter from the chip SRAM-B through the main control module for processing, and transmits the processed image data to the main control module after the gamma correction processing is completed.

[0020] S5, the main control module sends a completion signal to the image output module after receiving the corrected image data, and the image output module reads the processed image data and outputs the image data after digital-to-analog conversion by a digital-to-analog conversion chip.

[0021] Further, the chip SRAM-A and the chip SRAM-B are multiplexed in the process of data processing, so that the data input and output of different processing modules are integrated, that is, the data output area of the previous module is the data input area of the next module, and the storage resources are effectively saved.

[0022] Further, the process of filtering by the bilateral filter algorithm module in S3 is as follows:

[0023] S3.1, the bilateral filter algorithm module acquires the image data in the chip SRAM-A through the main control module, and buffers the acquired image data into four built-in FIFO units;

[0024] S3.2, in use, the current column of image data is directly read from the four FIFO units to form a filter window with a size of 5*5;

[0025] S3.3, the brightness difference between the neighborhood pixel points and the center pixel point in the current filter window of the image data is calculated;

[0026] S3.4, taking the luminance difference value as an address, inputting the luminance difference value obtained in S3.3, and searching a corresponding luminance proximity weight value W from a pre-stored luminance proximity weight value lookup table in one clock cycle r ;

[0027] S3.5, calculating the product of the spatial proximity weight value W of each pixel point in the current filtering window and the luminance proximity weight value W obtained in S3.4, to obtain a weight value W; and calculating the product P of the gray value of each pixel point in the current filtering window of the image data and the weight value W s r

[0028] S3.6, summing the product P of the gray value and the weight value W in the current filtering window of the image data

[0029] S3.7, accumulating the weight value W of each pixel point in the current filtering window of the image data; and using a divider to calculate the sum of the product P / the sum of the weight value W, to obtain the gray value of the pixel point after the bilateral filtering and output.

[0030] Further, the process of using the gamma correction processing module for gamma processing in S4 is:

[0031] S4.1, the gamma correction processing module reads the gray value of the pixel point obtained through the bilateral filtering from the chip SRAM-B through the main control module

[0032] S4.2, taking the gray value of the pixel point as an address, inputting the gray value of the pixel point obtained in S4.1, and searching a corresponding gamma function value from a pre-stored gamma function lookup table in one clock cycle; the value is the gray value of the pixel point obtained through the gamma correction processing.

[0033] The non-cooled infrared image enhancement system and method based on FPGA provided by the application is to take FPGA as a processing core, combine a bilateral filtering algorithm and a gamma correction algorithm, and enhance the infrared image in real time. The bilateral filtering algorithm considers two factors of the distance between pixel points and the gray value difference in the processing process, can clearly retain the original details of the image while enhancing the image effect, and achieves the purpose of edge preservation and noise removal. On this basis, the gamma correction algorithm is used to make the image after noise removal have a stronger contrast, and effectively improve the image quality. Meanwhile, the chip SRAM-A and the chip SRAM-B are multiplexed in the system framework, so that the whole system can process a large amount of data quickly by using less storage resources, thereby meeting the real-time requirement of the image processing system.

[0034] ​​Compared with the prior art, the application combines bilateral filtering algorithm and gamma correction algorithm, realizes enhancing image while clearly retaining image details, and achieves the purpose of edge-preserving enhancement. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 A FPGA-based non-cooled infrared image enhancement system architecture block diagram is provided in the application;

[0036] Figure 2 A FPGA-based non-cooled infrared image enhancement system architecture block diagram is provided in the application;

[0037] Figure 3 A FPGA-based non-cooled infrared image enhancement system architecture block diagram is provided in the application;

[0038] Figure 4 A FPGA-based non-cooled infrared image enhancement system architecture block diagram is provided in the application. DETAILED DESCRIPTION

[0039] In order to better understand the purpose, structure and function of the application, the application is described in detail below in combination with the drawings and examples.

[0040] As shown in Figure 1 The FPGA-based non-cooled infrared image enhancement system provided by the application includes an image acquisition control module, a non-cooled infrared detection core, a main control module, a chip SRAM-A, a chip SRAM-B, a bilateral filtering processing module, a gamma correction processing module, an image output module, a digital-analog conversion chip and a VGA output interface.

[0041] The non-cooled infrared detection core is connected with the image acquisition control module and is used to complete image data acquisition.

[0042] The image acquisition control module is connected with the main control module and is used to configure the working mode of the non-cooled infrared detection core according to requirements and receive and send image data collected by the non-cooled infrared detection core.

[0043] The main control module has a first end connected with the image acquisition control module, a second end connected with the chip SRAM-A, a third end connected with the bilateral filter processing module, a fourth end connected with the chip SRAM-B, a fifth end connected with the gamma correction processing module, and a sixth end connected with the image output module. The main control module is composed of an SRAM control unit and a flow control unit; the SRAM control unit is used for controlling the reading and writing of the chip SRAM-A and the chip SRAM-B, the first end of the SRAM control unit is connected with the image acquisition control module as the first end of the main control module, the second end of the SRAM control unit is connected with the chip SRAM-A as the second end of the main control module, the third end of the SRAM control unit is connected with the first end of the flow control unit, the fourth end of the SRAM control unit is connected with the chip SRAM-B as the fourth end of the main control module; the flow control unit is used for controlling the operation of the bilateral filter processing module and the gamma correction processing module, the second end of the flow control unit is connected with the bilateral filter processing module as the third end of the main control module, the third end of the flow control unit is connected with the gamma correction processing module as the fifth end of the main control module, and the fourth end of the flow control unit is connected with the image output module as the sixth end of the main control module.

[0044] The image output module is connected with the VGA output interface through a digital-analog conversion chip, is used for receiving the image data after the gamma correction, controlling the output time sequence of the image data, and making the image data adapt to the time sequence requirement of the output mode; the image data output by the image output module is processed through the digital-analog conversion chip after the digital-analog conversion, and is output through the VGA interface. In the embodiment, four FIFO units are arranged in the bilateral filter processing module, and the image data is buffered through the four FIFOs.

[0045] In the whole system architecture, each module adopts the pipeline design, and the throughput of the system is improved. In the data processing process, different storage areas are multiplexed, so that the data input and output of different processing modules are integrated, that is, the data output area of the previous module is the data input area of the next module, and the storage resources are effectively saved.

[0046] As shown in Figure 2 The method for image enhancement of the above-mentioned non-cooled infrared image enhancement system based on FPGA includes the following steps:

[0047] S1, first, the image acquisition control module configures the working mode of the non-cooled infrared detection core according to the requirement, and the configured working mode includes the image acquisition mode, pixel point precision and other information.

[0048] S2, after the configuration is completed, the non-cooled infrared detection core starts to acquire image data, and sends the acquired image data to the main control module through the image acquisition control module, and the main control module controls the writing of the image data into the chip SRAM-A for buffering.

[0049] S3, when a frame of original image data is written, the main control module sends a start signal to the bilateral filtering processing module; after receiving the start signal, the bilateral filtering processing module acquires the image data in the chip SRAM-A through the main control module, and then processes the image data by using the bilateral filtering algorithm. The expression of the bilateral filtering algorithm is as follows:

[0050]

[0051] In the formula, S x,y represents a 5x5 neighborhood of the center pixel point (x, y); w(i, j) represents a weighting coefficient; and g(i, j) represents a pixel point in the neighborhood. The weighting coefficient w(i, j) is composed of the product of a spatial proximity weight W s and a brightness proximity weight W r , and the expressions thereof are as follows:

[0052]

[0053]

[0054] In the formula, σ s represents a spatial proximity weight decay coefficient, and σ r represents a brightness proximity weight decay coefficient.

[0055] In the embodiment, a 5x5 window is used to perform bilateral filtering on the image, that is, 5 rows of data are read and processed each time. As shown in FIG. 3, the process in which the bilateral filtering algorithm module processes the image data by using the bilateral filtering algorithm includes the following steps. Figure 3

[0056] Further, the process of filtering by using the bilateral filtering algorithm module in S3 is as follows.

[0057] S3.1, the bilateral filtering algorithm module acquires the image data in the chip SRAM-A through the main control module, and caches the acquired image data into the four built-in FIFO units;

[0058] S3.2, in use, the current column of image data is directly read from the four FIFO units to form a 5x5 filtering window;

[0059] S3.3, the brightness difference between the neighborhood pixel points and the center pixel point in the current filtering window of the image data is calculated;

[0060] S3.4, the brightness difference value obtained in S3.3 is taken as an address, and the corresponding brightness proximity weight W r is found from the pre-stored brightness proximity weight lookup table in one clock cycle. ​

[0061] S3.5, calculate the spatial proximity weight W of each pixel point in the current filtering window s S3.4, calculate the brightness proximity weight W of each pixel point in the current filtering window r S3.4, calculate the brightness proximity weight W of each pixel point in the current filtering window

[0062] S3.6, sum the product P of the gray value and the weight W of each pixel point in the current filtering window of the image data.

[0063] S3.7, accumulate the weight W of each pixel point in the current filtering window of the image data; use the divider to calculate the sum of the product P / the sum of the weight W, to obtain the gray value of the pixel point after bilateral filtering and output.

[0064] It should be noted that in this embodiment, since the spatial proximity weight is only related to the position of the pixel point in the neighborhood and the number is small, it is defined as a parameter and directly written into the bilateral filtering processing module in code form for easy calling during operation.

[0065] S4, when the original image bilateral filtering processing is completed, the bilateral filtering processing module transmits the image data after bilateral filtering processing to the main control module and sends a completion signal. After receiving the completion signal, the main control module controls the image data after bilateral filtering processing to be cached in the chip SRAM-B, and sends a start signal to the gamma correction processing module. After receiving the start signal, the gamma correction processing module reads the image data after bilateral filtering processing from the chip SRAM-B through the main control module for processing, and transmits it to the main control module after the gamma correction processing is completed. The gamma correction module in this embodiment processes the image data after bilateral filtering, including the following steps:

[0066] S4.1, the gamma correction processing module reads the pixel point gray value obtained by the bilateral filtering processing from the chip SRAM-B through the main control module;

[0067] S4.2, input the pixel point gray value obtained in S4.1 with the pixel point gray value as the address, and search in the gamma function lookup table constructed in advance within 1 clock cycle to obtain the corresponding gamma function value and register the result; this value is the pixel point gray value obtained by the gamma correction processing.

[0068] S5, the main control module sends a completion signal to the image output module after receiving the corrected image data, and the image output module reads the processed image data and outputs it after digital-to-analog conversion through a digital-to-analog conversion chip.

[0069] The application provides a non-cooled infrared image enhancement system and method based on FPGA, which is based on a bilateral filtering algorithm and combined with a gamma correction algorithm, so that the image is enhanced and the image details are well preserved, the image is not distorted, and the edge-preserving enhancement is achieved.

[0070] To verify the feasibility of the non-cooled infrared image enhancement system based on FPGA, the application also provides application of the non-cooled infrared image enhancement system based on FPGA in a PCB.

[0071] As shown in Figure 4 The non-cooled infrared image enhancement system based on FPGA is integrated on the PCB, the PCB adopts a W25Q64JV chip with a size of 64Mbit, and is configured with an RS232 serial port module and a VGA module. The RS232 serial port module is used for interaction with a host computer to assist in debugging of the modules of the PCB. The host computer can communicate with the PCB through the RS232 serial port to transmit signals and data information to each other, so that the normal work of the PCB is ensured. The VGA module includes a digital-to-analog conversion chip and a VGA interface, the digital-to-analog conversion chip is connected with a VGA output interface of the non-cooled infrared image enhancement system based on FPGA, is used for receiving image data processed by gamma correction, and converting the image data into a video signal required by the VGA; the VGA interface is connected with an external display, is used for transmitting the video signal to the external display to display the final image result, so that the integration is completed.

[0072] It can be understood that the application is described through some embodiments, and those skilled in the art know that various changes or equivalent replacements can be made to the features and embodiments without departing from the spirit and scope of the application. In addition, the features and embodiments can be modified to adapt to specific conditions and materials under the guidance of the application without departing from the spirit and scope of the application. Therefore, the application is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of the application belong to the scope of protection of the application.

Claims

1. An FPGA-based uncooled infrared image enhancement system, comprising an uncooled infrared detection core, an image acquisition control module, a main control module, a chip SRAM-A, a chip SRAM-B, a bilateral filtering processing module, a gamma correction processing module, an image output module, a digital-to-analog conversion chip and a VGA output interface, characterized in that: the uncooled infrared detection core is connected with the image acquisition control module to complete image data acquisition; the image acquisition control module is connected with the main control module to configure the working mode of the uncooled infrared detection core according to requirements, and receive and send the image data collected by the uncooled infrared detection core; the main control module has a first end connected with the image acquisition control module, a second end connected with the chip SRAM-A, a third end connected with the bilateral filtering processing module, a fourth end connected with the chip SRAM-B, a fifth end connected with the gamma correction processing module, and a sixth end connected with the image output module; the main control module is used to control the running of the bilateral filtering processing module and the gamma correction processing module, and control the reading and writing of the image data in the chip SRAM-A and the chip SRAM-B; the chip SRAM-A is used to buffer the image data collected by the uncooled infrared detection core; the chip SRAM-B is used to buffer the image data processed by the bilateral filtering; the bilateral filtering processing module is used to obtain the image data provided by the image acquisition control and perform bilateral filtering processing; the gamma correction processing module is used to perform gamma correction processing on the image data processed by the bilateral filtering; the image output module is connected with the VGA output interface through the digital-to-analog conversion chip, and is used to receive the image data processed by the gamma correction, control the output timing of the image data, and make the image data adapt to the timing requirements of the output mode; the image data output by the image output module is converted into an analog signal through the digital-to-analog conversion chip, and is output through the VGA interface. The main control module comprises an SRAM control unit and a flow control unit; the SRAM control unit is used to control the reading and writing of the chip SRAM-A and the chip SRAM-B, and has a first end connected with the image acquisition control module as the first end of the main control module, a second end connected with the chip SRAM-A as the second end of the main control module, a third end connected with the first end of the flow control unit, and a fourth end connected with the chip SRAM-B as the fourth end of the main control module; the flow control unit is used to control the running of the bilateral filtering processing module and the gamma correction processing module, and has a second end connected with the bilateral filtering processing module as the third end of the main control module, a third end connected with the gamma correction processing module as the fifth end of the main control module, and a fourth end connected with the image output module as the sixth end of the main control module. ​ ​ ​ ​ ​ ​ 2. The FPGA-based uncooled infrared image intensifier system according to claim 1, characterized in that: ​ 3. The FPGA-based uncooled infrared image intensifier system according to claim 1, characterized in that: The bilateral filtering processing module is internally provided with four FIFO buffer units and a random access memory (RAM), the four FIFO buffer units are used for buffering the image data acquired from the chip SRAM-A by the main control module, the random access memory (RAM) pre-stores a brightness proximity weight lookup table constructed by taking the brightness difference value as the address and a gamma function lookup table constructed by taking the pixel point gray value as the address, the brightness proximity weight lookup table contains the address and the brightness proximity weight, and the gamma function lookup table contains the pixel point gray value and the gamma function value.

4. A FPGA-based uncooled infrared image enhancement method, which uses the FPGA-based uncooled infrared image enhancement system of claim 1 to perform image enhancement, characterized in that, The method comprises the following steps: S1, the image acquisition control module configures the working mode of the non-refrigeration type infrared detection core according to the requirement, the configured working mode comprises the image acquisition mode, pixel point precision and other information; S2, after the configuration is completed, the non-refrigeration type infrared detection core starts to acquire image data, and the acquired image data is sent to the main control module through the image acquisition control module, and the main control module controls the image data to be written into the chip SRAM-A for buffering; S3, after the image data is written, the main control module sends a start signal to the bilateral filtering processing module; after receiving the start signal, the bilateral filtering processing module acquires the image data in the chip SRAM-A through the main control module, and then processes the image data by using the bilateral filtering algorithm; S4, after the bilateral filtering processing is completed, the bilateral filtering processing module transmits the image data processed by the bilateral filtering to the main control module and sends a completion signal; after receiving the completion signal, the main control module controls the image data processed by the bilateral filtering to be buffered into the chip SRAM-B, and sends a start signal to the gamma correction processing module; after receiving the start signal, the gamma correction processing module reads the image data processed by the bilateral filtering from the chip SRAM-B through the main control module, processes the image data, and then transmits the processed image data to the main control module; S5, after receiving the corrected image data, the main control module sends a completion signal to the image output module, and the image output module reads the processed image data, converts the processed image data into an analog signal through a digital-to-analog conversion chip, and then outputs the analog signal.

5. The FPGA-based non-cooled infrared image enhancement method according to claim 4, characterized in that: In the process of data processing, the chip SRAM-A and the chip SRAM-B are multiplexed, so that the data input and output of different processing modules are integrated, that is, the data output area of the previous module is the data input area of the next module, and the storage resources are effectively saved.

6. The FPGA-based non-cooled infrared image enhancement method according to claim 5, characterized in that, The process of filtering processing by the bilateral filtering algorithm module in S3 is as follows: S3.1, the bilateral filtering algorithm module acquires the image data in the chip SRAM-A through the main control module, and buffers the acquired image data into the four built-in FIFO units; S3.2, in use, the current column of image data is directly read from the four FIFO units to form a filter window with a size of 5*5; S3.3, the brightness difference value between the neighborhood pixel points and the center pixel point in the current filter window of the image data is calculated; S3.4, input the luminance difference value obtained in S3.3 as the address, and find the corresponding luminance proximity weight value W from the pre-stored luminance proximity weight value lookup table within 1 clock cycle r ; S3.5, calculate the spatial proximity weight value W of each pixel in the current filtering window s S3.4, the product of the brightness proximity weight value W obtained in r S3.4, the product of the brightness proximity weight value W obtained in S3.5, calculate the spatial proximity weight value W of each pixel in the current filtering window S3.6, the sum of the products P of the gray values and the weights W in the current filter window of the image data is calculated; S3.7, the weights W of the pixel points in the current filter window of the image data are accumulated. The sum result of the product P and the sum result of the weight W are calculated by the divider to obtain a pixel gray value after bilateral filtering and output.

7. The FPGA-based non-cooled infrared image enhancement method according to claim 5, characterized in that, The gamma processing procedure of the gamma correction processing module in the S4 is: S4.1, the gamma correction processing module reads the pixel gray value obtained by the bilateral filtering from the chip SRAM-B through the main control module; S4.2, the pixel gray value obtained by S4.1 is input as an address, and the pixel gray value is searched from the pre-stored gamma function lookup table in one clock cycle to obtain a corresponding gamma function value; the value is the pixel gray value obtained by the gamma correction processing.

Images