Infrared camera image data exception processing method based on FPGA

Abstract

The invention discloses an infrared camera image data exception processing method based on an FPGA, and belongs to the technical field of FPGA and image processing. The method comprises the following steps: receiving original image data acquired by an infrared camera through an FPGA (Field Programmable Gate Array) and preprocessing the original image data; judging whether image exception processing is carried out or not, if yes, judging pixel gray values in the nth frame of image, and when the number of pixels with negative gray values exceeds a preset threshold value, calculating a negative gray value mean value of the nth frame of image; and averaging the mean values of the negative gray values of the (n-7) th frame to the nth frame again to obtain the mean value of the negative gray values of eight frames, and superposing the mean value of the negative gray values of the eight frames to each pixel gray value of the (n + 1) th frame, and outputting final image data. According to the invention, the characteristics of rich logic resources, high speed and high performance of the FPGA are fully exerted, and the real-time performance and reliability of image data processing are guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical fields of FPGA and image processing, and particularly relates to a method for processing abnormal infrared camera image data based on FPGA. Background Art

[0002] Current methods for processing abnormal infrared camera image data mainly include traditional image processing (such as filtering, interpolation), deep learning (such as GAN, autoencoder), multimodal fusion (such as visible light-infrared alignment), physical model correction (such as non-uniformity calibration), and temporal analysis (such as optical flow method). However, traditional methods rely on manual parameter adjustment and are prone to losing details; deep learning requires a large amount of labeled data and has a high computational cost; multimodal fusion has complex hardware and difficult registration; physical models are difficult to handle dynamic noise; and temporal analysis has a high false detection rate in dynamic scenes. Generally speaking, existing technologies generally face problems such as poor adaptability, insufficient real-time performance, and weak small-sample generalization ability, especially being limited in complex environments or high-precision scenarios.

[0003] Field Programmable Gate Arrays (FPGA) is a programmable signal processing device with rich logic resources that can be reprogrammed. Users can independently change the configuration information according to design requirements to define functions.

[0004] Image processing is a technology for analyzing and operating on images, aiming to improve the quality of images or extract useful information. It is widely used in fields such as medical imaging, autonomous driving, surveillance systems, industrial inspection, and face recognition. In these applications, real-time processing and efficient computing are crucial, which is exactly where FPGA (Field Programmable Gate Array) has its advantages.

[0005] FPGA has a high degree of parallel processing ability and customizability, and can implement complex image processing algorithms at the hardware level, such as edge detection, image filtering, and feature extraction. This enables FPGA to perform excellently when processing large-scale data, significantly reducing processing latency and increasing data throughput. At the same time, the parallel architecture of FPGA allows multiple processing units to work simultaneously, adapting to various image processing tasks, thereby improving the overall efficiency of the system. In addition, FPGA can be programmed and configured according to specific requirements, allowing developers to optimize performance in different application scenarios without the need to redesign the hardware.

[0006] Compared with traditional processors, FPGA also has certain advantages in terms of power consumption and cost. Especially in applications that run for a long time and have high real-time requirements, FPGA can provide higher cost performance. In summary, FPGA provides a flexible, efficient, and economical solution for image processing applications, making it an ideal choice for implementing image processing algorithms. Summary of the Invention

[0007] To solve the above technical problems, the present invention provides a method for processing abnormal infrared camera image data based on FPGA, which gives full play to the characteristics of rich logic resources, high speed and high performance of FPGA, ensures the real-time and reliability of image data processing, and provides more space for subsequent image processing.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] A method for processing abnormal infrared camera image data based on FPGA includes:

[0010] Step 1: Receive the original image data collected by the infrared camera through the FPGA and perform preprocessing;

[0011] Step 2: Judge whether to perform image abnormality processing according to the control instruction sent by the host computer. If necessary, execute Step 3;

[0012] Step 3: Judge the pixel gray values in the nth frame of the image. When the number of pixels with negative gray values exceeds a preset threshold, calculate the average value of the negative gray values of the nth frame of the image;

[0013] Step 4: Average the average values of the negative gray values of the 8 frames again by averaging the average values of the negative gray values from the (n - 7)th frame to the nth frame, and superimpose the average value of the 8 frames of negative gray values on each pixel gray value of the (n + 1)th frame to output the final image data.

[0014] In a second aspect, the present invention provides an electronic device, including: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method for processing abnormal infrared camera image data based on FPGA.

[0015] In a third aspect, the present invention provides a computer-readable storage medium, on which executable instructions are stored, and when the instructions are executed by a processor, the processor can implement the aforementioned method for processing abnormal infrared camera image data based on FPGA.

[0016] The beneficial effects of the present invention are as follows:

[0017] The present invention is an indispensable and important part of the image processing algorithm, which can better make up for the abnormal phenomenon of image data under weak light after dark field correction; processes different pixel points of the same image, and better preserves the original information of the image, providing a better data source for subsequent image processing and reality.

[0018] When there are three or more pixel gray - scale value errors in the image, the host computer sends instructions to control whether the FPGA performs pixel processing. This method can effectively avoid over - processing of the entire frame of the image due to accidental image data errors and the algorithm under the condition of a dark background and bright targets.

[0019] Aiming at the problem that the mean difference of abnormal gray - scale values in each frame of data is relatively large, the present invention uses the mean value of negative gray - scale values from the (n - 7)th to the nth frame and superimposes it on the gray - scale value of each pixel in the (n + 1)th frame, significantly reducing the overall gray - scale value difference between adjacent frames, enhancing the usability of the algorithm, and providing convenience for subsequent image processing.

[0020] The present invention does not rely on manual parameter adjustment and has strong autonomy. The parameters used all originate from camera parameters and camera - acquired data, which means that once the system is determined, the parameters do not need to be adjusted.

[0021] The present invention uses the FPGA to perform data processing at the image input end, which has the characteristic of good real - time performance and provides a basis for subsequent image processing and applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is the schematic diagram of a method for processing abnormal image data of an infrared camera based on FPGA according to the present invention;

[0023] Figure 2 is the flow chart of a method for processing abnormal image data of an infrared camera based on FPGA according to the present invention;

[0024] Figure 3 is the original image output when abnormal data of the infrared camera appears;

[0025] Figure 4 is the image output after being processed by the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The present invention will be further described below with reference to the drawings and embodiments.

[0027] As Figure 1 and Figure 2 shown, a method for processing abnormal image data of an infrared camera based on FPGA according to the present invention has the following specific steps:

[0028] Step 1: Receive the original image data collected by the infrared camera through the FPGA and perform pre - processing, including: The image acquisition device receives the original image data collected by the infrared camera through Cam - link (taking the 14 - bit infrared camera Cred - 2 as an example in this article), sends it to the FPGA, and the FPGA arranges and organizes the original image data in a 4 - tap, 16 - bit image format.

[0029] Step 2: Determine whether to perform image anomaly processing according to the control instructions sent by the host computer. If necessary, execute Step 3, including: receiving the control command sent by the image display device (i.e., the host computer) through the optical fiber to the FPGA, and then the FPGA determines whether image anomaly data processing is required according to the pre-determined protocol for the instruction; when anomaly data processing is required, execute the next step, and when it is not required, directly output the original image data to the image display device;

[0030] Step 3: Judge the pixel gray values in the nth frame of image. When the number of pixels with negative gray values exceeds the preset threshold, calculate the average value of the negative gray values of the nth frame of image; including:

[0031] Step 3.1: Judge the gray values of each pixel point in the nth frame of image, and determine the pixel data with the highest bit (the 16th bit) of the pixel gray value being 1 as negative;

[0032] Step 3.2: Count the pixel data with negative gray values in the nth frame of image to obtain the negative number p n ;

[0033] Step 3.3: Judge whether to process the nth frame of image according to the value of the negative number p n ; if p n is less than or equal to 3, do not perform algorithm processing on the image, retain the original image data and output it; if p n is greater than 3, calculate the average value of the negative gray values of the nth frame of image; including:

[0034] a. Accumulate and sum up the gray values corresponding to all negatives to obtain the total sum S of the negative gray values of the nth frame of image n ;

[0035] b. Use a divider to divide the total sum S of the negative gray values n by the negative number p n and the negative gray value to obtain the average value of the negative gray values in the nth frame, and get the average value a of the negative gray values in the nth frame n ;

[0036] Step 4: Average the average values of the negative gray values from the (n - 7)th frame to the nth frame again to obtain the average value of the 8-frame negative gray values, and superimpose the average value of the 8-frame negative gray values on each pixel gray value of the (n + 1)th frame, and output the final image data; including:

[0037] Step 4.1: Average the average values of the negative gray values from the (n - 7)th frame to the nth frame again to obtain the average value A of the 8-frame negative gray values (n-7 ~n) ;

[0038] Step 4.2: Take the absolute value |A (n-7 ~n) of the average value A of the negative gray values of the 8 frames(n-7 ~n) Overlay on each pixel of the (n + 1)-th frame;

[0039] Step 4.3: Further judge the gray values of the overlaid image, and process the pixel points with still abnormal gray values to obtain the final image data. Among them, the further judgment is divided into three different situations:

[0040] Step 4.3.1: If the most significant bit of the gray value of the overlaid pixel is still 1, it is judged that the gray value of the overlaid pixel point is still negative. To ensure normal image display, in this case, set the gray value of the overlaid pixel point to 0.

[0041] Step 4.3.2: If the gray value of the overlaid pixel point is greater than 16383 (X03FF), set the gray value of the overlaid pixel point to 16383 (X03FF);

[0042] Step 4.3.3: If the gray value of the overlaid pixel point is less than 16383, the overlaid value remains unchanged;

[0043] Step 4.4: Output the final image data.

[0044] See Figure 3 the original image output example showing the abnormal situation of the infrared camera data. After being processed by the method of the present invention, the output as shown in Figure 4 is obtained, and it can be clearly shown that the noise is greatly reduced, verifying the effectiveness of the present invention.

[0045] In a second aspect, the present invention provides an electronic device, including: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the foregoing method for processing abnormal image data of an infrared camera based on FPGA.

[0046] In a third aspect, the present invention provides a computer-readable storage medium, on which executable instructions are stored, and when the instructions are executed by a processor, the processor can implement the foregoing method for processing abnormal image data of an infrared camera based on FPGA.

[0047] The above specific embodiments further elaborate the purpose, technical solution and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for abnormal processing of infrared camera image data based on FPGA, characterized in that: include: Step 1: Receive the raw image data collected by the infrared camera through FPGA and perform preprocessing; Step 2: Determine whether to perform image abnormality processing according to the control command sent by the host computer, and execute step 3 if necessary; Step 3, judging the grayscale values ​​of pixels in the n-th frame image, and when the number of pixels with negative grayscale values ​​exceeds a preset threshold, calculating the mean of the negative grayscale values ​​of the n-th frame image; Step 4: average the negative grayscale values ​​from the n-7th frame to the nth frame again to obtain the average of the negative grayscale values ​​of 8 frames, and superimpose the average of the negative grayscale values ​​of the 8 frames on the grayscale value of each pixel in the n+1th frame to output the final image data.

2. The method for abnormal processing of infrared camera image data based on FPGA according to claim 1, characterized in that: In the step 1, the preprocessing includes that the FPGA arranges and organizes the original image data according to the 4tap, 16-bit image format.

3. The method for abnormal processing of infrared camera image data based on FPGA according to claim 1, characterized in that: In step 2, the FPGA receives the control command sent by the host computer through the optical fiber, and determines whether abnormal image data processing is required according to the preset protocol. When abnormal data processing is required, the next step is executed, and when it is not required, the original image data is directly output to the host computer.

4. The method for abnormal processing of infrared camera image data based on FPGA according to claim 1, characterized in that: The step 3 comprises: Step 3.1, determine the gray value of each pixel in the nth frame image, and determine the pixel data with the highest gray value of 1 as a negative number; Step 3.2: Count the pixel data with negative grayscale values ​​in the nth frame image to obtain the number of negative numbers p n ; Step 3.3: According to the number of negative numbers p n The value determines whether to process the nth frame image; if p n Less than or equal to 3, retain the original image data and output; if p n If it is greater than 3, the mean of the negative grayscale values ​​of the nth frame image is calculated.

5. The method for abnormal processing of infrared camera image data based on FPGA according to claim 3 is characterized in that: The step 3.3 comprises: The grayscale values ​​corresponding to all negative numbers of the n-th frame image are accumulated and summed to obtain the total negative grayscale value S of the n-th frame image. n ; Use a divider to sum the negative gray values ​​S n Divide by negative quantity p n The negative grayscale value and the negative grayscale value are used to obtain the average negative grayscale value in the nth frame, and the average negative grayscale value a in the nth frame is obtained. n .

6. The method for abnormal processing of infrared camera image data based on FPGA according to claim 1, characterized in that: The step 4 comprises: Step 4.1: average the negative grayscale values ​​from the n-7th frame to the nth frame again to obtain the average value A of the negative grayscale values ​​of the 8 frames. (n-7 ~n) ; Step 4.2: Take the average value A of the negative grayscale values ​​of 8 frames (n-7 ~n) The absolute value of |A (n-7 ~n) |Superimposed on every pixel of the n+1th frame; Step 4.3, judging the grayscale value of the superimposed image, and processing the pixel points whose grayscale value is still abnormal to obtain the final image data; Step 4.4: Output the final image data.

7. The method for abnormal processing of infrared camera image data based on FPGA according to claim 6, characterized in that: In step 4.3, judging the grayscale value of the superimposed image includes: Step 4.3.1, if the highest bit of the grayscale value of the superimposed pixel is still 1, it is determined that the grayscale value of the superimposed pixel is still a negative number; Step 4.3.2, if the grayscale value of the superimposed pixel is greater than 16383, set the grayscale value of the superimposed pixel to 16383; Step 4.3.3: If the grayscale value of the superimposed pixel is less than 16383, the superimposed value remains unchanged.

8. The method for abnormal processing of infrared camera image data based on FPGA according to claim 7, characterized in that: In step 4.3.1, the grayscale value of the superimposed pixel is set to 0.

9. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; Wherein, when one or more programs are executed by the one or more processors, the one or more processors implement the FPGA-based infrared camera image data exception processing method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: Executable instructions are stored thereon, and when the instructions are executed by the processor, the processor can implement the FPGA-based infrared camera image data exception processing method described in any one of claims 1-8.

Images