A method for abnormal processing of infrared camera image data based on FPGA

Abstract

The present invention discloses a method for processing image data anomalies of an infrared camera based on FPGA, which belongs to the field of FPGA and image processing technology. The method comprises: receiving raw image data collected by an infrared camera through FPGA and performing preprocessing; judging whether to perform image anomaly processing, and if necessary, judging the pixel grayscale value in the n-th frame image, and when the number of pixels with negative grayscale values ​​exceeds a preset threshold, calculating the mean of the negative grayscale values ​​of the n-th frame image; averaging the mean of the negative grayscale values ​​of the n-7th frame to the n-th frame again to obtain the mean of the negative grayscale values ​​of 8 frames, and superimposing the mean of the negative grayscale values ​​of the 8 frames on the grayscale value of each pixel in the n+1th frame, and outputting the final image data. The present invention gives full play to the characteristics of rich FPGA logic resources, high speed and high performance, and ensures the real-time and reliability of image data processing.

Description

Technical Field

[0001] The present invention belongs to the field of FPGA and image processing technology, and particularly relates to an FPGA-based infrared camera image data abnormality processing method. Background Art

[0002] Current methods for addressing anomalies in infrared camera image data primarily include traditional image processing (such as filtering and interpolation), deep learning (such as GANs and autoencoders), multimodal fusion (such as visible-infrared alignment), physical model correction (such as non-uniformity calibration), and time series analysis (such as optical flow). However, traditional methods rely on manual parameter adjustment and are prone to losing details; deep learning requires large amounts of annotated data and is computationally expensive; multimodal fusion hardware is complex and registration is difficult; physical models struggle to cope with dynamic noise; and time series analysis suffers from high false positive rates in dynamic scenes. Overall, existing technologies generally suffer from poor adaptability, insufficient real-time performance, and weak generalization capabilities for small sample sizes, particularly limiting their performance in complex environments or high-precision scenes.

[0003] Field Programmable Gate Arrays (FPGA) are programmable signal processing devices with rich logic resources and reprogrammable. Users can independently change configuration information to define functions according to design requirements.

[0004] Image processing is the analysis and manipulation of images to improve their quality or extract useful information. It is widely used in fields such as medical imaging, autonomous driving, surveillance systems, industrial inspection, and facial recognition. In these applications, real-time processing and efficient computing are crucial, which is precisely the advantage of FPGAs (field programmable gate arrays).

[0005] FPGAs possess a high degree of parallel processing capability and customizability, enabling the implementation of complex image processing algorithms such as edge detection, image filtering, and feature extraction at the hardware level. This enables FPGAs to excel at processing large amounts of data, significantly reducing processing latency and increasing data throughput. Furthermore, the FPGA's parallel architecture allows multiple processing units to operate simultaneously, adapting to a variety of image processing tasks and thereby improving the overall efficiency of the system. Furthermore, FPGAs can be programmed and configured according to specific needs, allowing developers to optimize performance for different application scenarios without redesigning the hardware.

[0006] Compared to traditional processors, FPGAs also offer advantages in power consumption and cost, especially in applications that run for extended periods and have strict real-time requirements, where FPGAs offer a higher cost-performance ratio. In summary, FPGAs provide a flexible, efficient, and economical solution for image processing applications, making them an ideal choice for implementing image processing algorithms. Summary of the Invention

[0007] To solve the above technical problems, the present invention provides an FPGA-based infrared camera image data anomaly processing method, which fully utilizes the rich logic resources, high speed and high performance of FPGA, ensures the real-time and reliability of image data processing, and provides more space for subsequent image processing.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] A method for processing abnormal image data of an infrared camera based on FPGA, comprising:

[0010] Step 1: Receive the raw image data collected by the infrared camera through FPGA and perform preprocessing;

[0011] Step 2: Determine whether to perform image abnormality processing according to the control instruction sent by the host computer. If necessary, execute step 3;

[0012] Step 3: Determine the grayscale values ​​of pixels in the n-th frame image. When the number of pixels with negative grayscale values ​​exceeds a preset threshold, calculate the average of the negative grayscale values ​​of the n-th frame image.

[0013] Step 4: average the negative grayscale values ​​from the n-7th frame to the nth frame again to obtain the average of the negative grayscale values ​​of 8 frames, and superimpose the average of the negative grayscale values ​​of the 8 frames on the grayscale value of each pixel in the n+1th frame to output the final image data.

[0014] In a second aspect, the present invention provides an electronic device comprising: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned FPGA-based infrared camera image data anomaly processing method.

[0015] In a third aspect, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the aforementioned FPGA-based infrared camera image data anomaly processing method.

[0016] The beneficial effects of the present invention are:

[0017] The present invention is an indispensable and important part of the image processing algorithm. It can better compensate for the abnormal phenomenon of image data under weak light after dark field correction. It processes different pixel points of the same image, better preserves the original information of the image, and provides a better data source for subsequent processing and reality of the image.

[0018] In the present invention, when three or more pixel grayscale value errors occur in the image, the host computer sends instructions to control the FPGA to control whether to perform pixel processing. This method can effectively avoid excessive processing of the entire frame image by the algorithm due to occasional image data errors and the condition of dark background and bright target.

[0019] To address the problem of relatively large mean differences in abnormal grayscale values ​​in each frame of data, the present invention uses the mean of negative grayscale values ​​from frames n-7 to n to be superimposed on the grayscale value of each pixel in frame n+1, significantly reducing the overall grayscale value difference between adjacent frames, enhancing the usability of the algorithm, and providing convenience for subsequent image processing.

[0020] The present invention does not rely on manual parameter adjustment and has strong autonomy. The parameters used are all derived from camera parameters and camera acquisition data, which means that once the system is determined, the parameters do not need to be adjusted.

[0021] The present invention uses FPGA to process data at the image input end, has the characteristics of good real-time performance, and provides a basis for subsequent image processing and application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of a method for abnormal processing of infrared camera image data based on FPGA in the present invention;

[0023] Figure 2 This is a flow chart of an FPGA-based infrared camera image data exception processing method of the present invention;

[0024] Figure 3 Output of the original image where the infrared camera data is abnormal;

[0025] Figure 4 It is the image output after being processed by the method of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described below with reference to the accompanying drawings and examples.

[0027] like Figure 1 and Figure 2 As shown, the present invention provides an FPGA-based infrared camera image data abnormality processing method, the specific steps are as follows:

[0028] Step 1: Receive and preprocess the raw image data captured by the infrared camera through the FPGA. This includes: The image acquisition device receives the raw image data captured by the infrared camera through Cam-Link (this article uses the 14-bit infrared camera Cred-2 as an example) and sends it to the FPGA. The FPGA arranges and organizes the raw image data according to the 4-tap, 16-bit image format.

[0029] Step 2: Determine whether to perform image abnormality processing based on the control command sent by the host computer. If necessary, execute step 3, which includes: receiving the control command sent by the image display device (i.e., the host computer) via optical fiber to the FPGA, and then the FPGA determines whether image abnormality data processing is required based on a pre-defined protocol; if abnormal data processing is required, execute the next step; if not, directly output the original image data to the image display device;

[0030] Step 3: determining the grayscale values ​​of pixels in the n-th frame image, and when the number of pixels with negative grayscale values ​​exceeds a preset threshold, calculating the mean of the negative grayscale values ​​of the n-th frame image; including:

[0031] Step 3.1, determine the grayscale value of each pixel in the nth frame image, and determine the pixel data with the highest grayscale value (16th bit) being 1 as a negative number;

[0032] Step 3.2: Count the pixel data with negative grayscale values ​​in the nth frame image and get the number of negative numbers p n ;

[0033] Step 3.3, according to the number of negative numbers p n The value determines whether to process the nth frame image; if p n If p is less than or equal to 3, the image will not be processed by the algorithm, and the original image data will be retained and output; if p n If it is greater than 3, the mean of the negative grayscale values ​​of the n-th frame image is calculated; including:

[0034] a. Accumulate and sum the grayscale values ​​corresponding to all negative numbers to obtain the total negative grayscale value S of the nth frame image. n ;

[0035] b. Use a divider to sum the negative grayscale values ​​S n Divide by negative quantity p n The negative grayscale value in the nth frame is calculated by summing the negative grayscale value and obtaining the average negative grayscale value a in the nth frame. n ;

[0036] Step 4: averaging the mean of the negative grayscale values ​​from the n-7th frame to the nth frame to obtain the mean of the negative grayscale values ​​of 8 frames, and superimposing the mean of the negative grayscale values ​​of the 8 frames on the grayscale value of each pixel in the n+1th frame to output the final image data; including:

[0037] Step 4.1: average the negative grayscale values ​​from the n-7th frame to the nth frame to obtain the average value A of the negative grayscale values ​​of the 8 frames. (n-7 ~n) ;

[0038] Step 4.2, take the average negative grayscale value A of 8 frames (n-7 ~n) The absolute value of |A(n-7 ~n) |Superimposed on every pixel of the n+1th frame;

[0039] Step 4.3: Further judge the grayscale value of the superimposed image and process the pixels whose grayscale values ​​are still abnormal to obtain the final image data. The further judgment is divided into three different situations:

[0040] Step 4.3.1: If the highest bit of the superimposed pixel grayscale value is still 1, the grayscale value of the superimposed pixel is still negative. To ensure normal image display, the grayscale value of the superimposed pixel is set to 0 in this case.

[0041] Step 4.3.2: If the grayscale value of the superimposed pixel is greater than 16383 (X03FF), set the grayscale value of the superimposed pixel to 16383 (X03FF);

[0042] Step 4.3.3: If the grayscale value of the superimposed pixel is less than 16383, the superimposed value remains unchanged;

[0043] Step 4.4: Output the final image data.

[0044] See also Figure 3 The original image output example of the abnormal infrared camera data is shown in FIG. 1 . After being processed according to the method of the present invention, the following is obtained: Figure 4 The output shown clearly shows that the noise is greatly reduced, verifying the effectiveness of the present invention.

[0045] In a second aspect, the present invention provides an electronic device comprising: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned FPGA-based infrared camera image data anomaly processing method.

[0046] In a third aspect, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the aforementioned FPGA-based infrared camera image data anomaly processing method.

[0047] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for processing abnormal image data of an infrared camera based on FPGA, characterized in that: include: Step 1: Receive the raw image data collected by the infrared camera through FPGA and perform preprocessing; Step 2: Determine whether to perform image abnormality processing according to the control instruction sent by the host computer. If necessary, execute step 3; Step 3: Determine the grayscale values ​​of pixels in the n-th frame image. When the number of pixels with negative grayscale values ​​exceeds a preset threshold, calculate the average of the negative grayscale values ​​of the n-th frame image. include: Step 3.1, determine the grayscale value of each pixel in the nth frame image, and determine the pixel data with the highest grayscale value of 1 as a negative number; Step 3.2: Count the pixel data with negative grayscale values ​​in the nth frame image and get the number of negative numbers p n ; Step 3.3, according to the number of negative numbers p n The value determines whether to process the nth frame image; if p n Less than or equal to 3, retain the original image data and output; if p n If it is greater than 3, the mean of the negative grayscale values ​​of the n-th frame image is calculated; Step 4: Averaging the mean of the negative grayscale values ​​from the n-7th frame to the nth frame again to obtain the mean of the negative grayscale values ​​of 8 frames, and superimposing the mean of the negative grayscale values ​​of the 8 frames on the grayscale value of each pixel in the n+1th frame to output the final image data, including: Step 4.1: average the negative grayscale values ​​from the n-7th frame to the nth frame to obtain the average value A of the negative grayscale values ​​of the 8 frames. (n-7 ~n) ; Step 4.2: Take the average value A of the negative grayscale values ​​of 8 frames (n-7 ~n) The absolute value of |A (n-7 ~n) |Superimposed on every pixel of the n+1th frame; Step 4.3: judge the grayscale value of the superimposed image and process the pixels whose grayscale values ​​are still abnormal to obtain the final image data; Step 4.4: Output the final image data.

2. The method for abnormal processing of infrared camera image data based on FPGA according to claim 1, characterized in that: In the step 1, the pre-processing includes arranging and arranging the original image data according to the 4-tap, 16-bit image format by the FPGA.

3. The method for abnormal processing of infrared camera image data based on FPGA according to claim 1, characterized in that: In step 2, the FPGA receives the control command sent by the host computer through the optical fiber, and determines whether image abnormality data processing is required according to the preset protocol. When abnormal data processing is required, the next step is executed. When it is not required, the original image data is directly output to the host computer.

4. The method for abnormal processing of infrared camera image data based on FPGA according to claim 1, characterized in that: The step 3.3 includes: The grayscale values ​​corresponding to all negative numbers of the n-th frame image are accumulated and summed to obtain the total negative grayscale value S of the n-th frame image. n ; Use the divider to sum the negative grayscale values ​​S n Divide by negative quantity p n Get the average value of negative grayscale values ​​in the nth frame and get the average value of negative grayscale values ​​in the nth frame a n .

5. The method for abnormal processing of infrared camera image data based on FPGA according to claim 1, characterized in that: In step 4.3, determining the grayscale value of the superimposed image includes: Step 4.3.1: If the highest bit of the superimposed pixel grayscale value is still 1, it is determined that the grayscale value of the superimposed pixel is still negative; Step 4.3.2: If the grayscale value of the superimposed pixel is greater than 16383, set the grayscale value of the superimposed pixel to 16383; Step 4.3.3: If the grayscale value of the superimposed pixel is less than 16383, the superimposed value remains unchanged.

6. The method for processing abnormal image data of an infrared camera based on FPGA according to claim 5, characterized in that: In step 4.3.1, the grayscale value of the superimposed pixel is set to 0.

7. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs; When one or more programs are executed by the one or more processors, the one or more processors implement the FPGA-based infrared camera image data exception processing method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that Executable instructions are stored thereon, and when the instructions are executed by the processor, the processor can implement the FPGA-based infrared camera image data abnormality processing method described in any one of claims 1-6.

Images