Infrared image flicker processing method, system, equipment and medium

Abstract

The invention provides an infrared image flicker processing method, system and device and a medium. The method comprises the steps of generating a blocking piece mean value based on blocking piece image data of continuous N frames obtained during single-point correction; processing the histograms of the two-time single-point correction gap images according to the separation blade mean value to generate a statistical histogram of the images; performing first filtering processing on the statistical histogram to generate a first filtering result of the histogram; after the first filtering result is calculated in an accumulated mode, a gray mapping table is calculated, second filtering processing is carried out, and a second filtering result of the gray mapping table is generated; and performing time domain filtering processing on the second filtering result to realize flicker processing of the infrared image. According to the infrared image flicker processing method, the flicker phenomenon of the infrared image can be effectively reduced, and the stability and the visual effect of the image are improved.

Description

Technical Field

[0001] The embodiments of this application relate to the field of data processing, and more particularly to an infrared image flicker processing method, system, device, and medium. Background Technology

[0002] Infrared imaging technology plays a vital role in modern technology, with widespread applications in military reconnaissance, industrial inspection, medical imaging, environmental monitoring, and intelligent transportation. Infrared images can provide thermal radiation information of targets under low-light or complex environmental conditions, making them particularly important for nighttime surveillance and applications in adverse weather conditions. However, infrared images are often plagued by flickering in practical applications, a problem primarily stemming from factors such as sensor noise, changes in ambient light, equipment circuit jitter, and temperature drift.

[0003] Flickering causes visual instability in images, affecting subsequent image processing and analysis tasks such as target recognition, image enhancement, and edge detection. To mitigate the impact of flickering on infrared image quality, researchers have proposed various algorithms and techniques, including time-domain filtering, spatial-domain filtering, frequency-domain analysis, and image sequence processing.

[0004] Temporal filtering reduces flicker by averaging or weighted averaging consecutive image frames, but this method may result in loss of image detail. Spatial domain filtering reduces noise points and improves image quality through smoothing, but it may not be ideal for handling dynamic scenes. Frequency domain analysis converts the image signal to the frequency domain using Fourier transform to remove high-frequency flicker components, but this method has high computational complexity and is not suitable for highly integrated small devices and chip-level implementation.

[0005] In recent years, with the development of hardware technology, FPGAs have become an ideal choice for solving real-time image processing problems due to their parallel processing capabilities and programmability. FPGAs can implement complex image processing algorithms through hardware acceleration, thereby improving processing speed and efficiency. Although FPGAs have been applied in the field of image processing, FPGA solutions for the infrared image flicker problem are still relatively few. Therefore, developing an efficient FPGA-based algorithm to improve the infrared image flicker problem is of great significance. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention proposes an infrared image flicker processing method, system, device, and medium. The infrared image flicker processing method of this invention can effectively reduce flickering in infrared images, improving image stability and visual effects. Through hardware acceleration using an FPGA, real-time processing capabilities are achieved, meeting the need for efficient processing of large amounts of image data. Furthermore, the parallel processing capability of the FPGA significantly improves the algorithm's efficiency and reduces system power consumption.

[0007] In a first aspect, the present invention proposes an infrared image flicker processing method, which specifically includes the following steps: The mean value of the baffle is generated based on the baffle image data of N consecutive frames obtained during single-point correction. A statistical histogram of the image is generated by processing the image histogram between two single-point corrections based on the mean value of the baffle. The statistical histogram is subjected to a first filtering process to generate a first filtered result of the histogram; After accumulating the first filtering result, calculate the grayscale mapping table and perform the second filtering process to generate the second filtering result of the grayscale mapping table; The second filtering result is subjected to time-domain filtering to achieve the flickering processing of the infrared image.

[0008] Furthermore, the mean value of the barrier is generated based on the barrier image data of N consecutive frames obtained during single-point correction. Specifically, this includes generating the mean value of the barrier based on the number of rows, columns, and pixel values ​​of the barrier image. , represented as:

[0009] in, The row number of the baffle image. The column number of the baffle image. This indicates the number of samples collected when the baffle is closed. Line number Column pixel values.

[0010] Furthermore, the step of generating the average value of the baffle based on the baffle image data of N consecutive frames obtained during single-point correction also includes: recalculating and updating the average value of the baffle each time a single-point correction is performed, and using the average value of the baffle obtained from the previous single-point correction for calculation between two single-point corrections.

[0011] Furthermore, generating a statistical histogram of the image by processing the histogram based on the mean of the baffles specifically includes: generating a statistical histogram of the image by performing length and center position-based constraint processing on the histogram based on the mean of the baffles. , represented as:

[0012]

[0013] in, To restrict the center position of the processed histogram, To limit the length of the processed histogram, This represents the raw pixel values ​​of the infrared image of the actual scene captured with the baffle open.

[0014] Furthermore, the statistical histogram is subjected to a first filtering process to generate the first filtered result of the histogram, which is expressed as:

[0015] The calculation results of the histogram of the previous frame of the current image.

[0016] Furthermore, after accumulating the first filtering result, calculating the grayscale mapping table and performing a second filtering process to generate the second filtering result of the grayscale mapping table specifically includes: The cumulative calculation of the first filtering result is performed according to the following formula to obtain the cumulative calculation result. , represented as:

[0017] A grayscale mapping table is calculated based on cumulative calculation results, preset image contrast, and preset image brightness. , represented as:

[0018] The grayscale map is subjected to a second filtering process to generate the second filtering result of the grayscale map, which is represented as follows:

[0019] in, Indicates the preset image contrast. Indicates the preset image brightness. This represents the calculation result of the grayscale mapping table for the Nth frame.

[0020] Furthermore, the time-domain filtering process is performed using a time-domain denoising model, which is specifically as follows:

[0021]

[0022] in, This represents the array of time-domain noise reduction thresholds. This represents the image data from the previous frame. This represents the image data of the current frame. This represents the image data after time-domain filtering. It represents the absolute value of the difference between the image data of the current frame and the previous frame.

[0023] In a second aspect, the present invention also proposes an infrared image flickering processing system for implementing the infrared image flickering processing method as described in the first aspect. The infrared image flickering processing system includes: The generation module is used to generate the average value of the baffle based on the baffle image data of N consecutive frames obtained during single-point correction. The flicker processing module is used to process the histogram of the image between two single-point corrections based on the mean value of the baffle to generate a statistical histogram of the image, perform a first filtering process on the statistical histogram to generate a first filtering result of the histogram, accumulate and calculate the first filtering result, calculate the gray-level mapping table and perform a second filtering process to generate a second filtering result of the gray-level mapping table, and perform time-domain filtering process on the second filtering result to realize flicker processing of the infrared image. The display module is used to display the image processed by the flickering processing module in real time.

[0024] Thirdly, the present invention also proposes an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0025] Fourthly, the present invention also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect of the present invention.

[0026] The beneficial effects of this invention are as follows: The algorithm of this invention can effectively reduce the flickering phenomenon of infrared images, improve image stability and visual effect. Through FPGA hardware acceleration, real-time processing capability is achieved, meeting the need for efficient processing of large amounts of image data. Furthermore, the parallel processing capability of the FPGA significantly improves the efficiency of the algorithm and reduces the power consumption of the system. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the infrared image flicker processing method proposed in this invention; Figure 2 This is a schematic diagram of the infrared image flicker processing system proposed in this invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] This invention proposes a method for processing infrared image flicker, such as... Figure 1 As shown, the infrared image flicker processing method includes the following steps: S101. Generate the mean value of the baffle based on the baffle image data of N consecutive frames obtained during single-point correction; S102. Generate a statistical histogram of the image by processing the histogram of the image between two single-point corrections based on the mean value of the baffle; S103. Perform a first filtering process on the statistical histogram to generate a first filtered result of the histogram; S104. After accumulating the first filtering result, calculate the grayscale mapping table and perform the second filtering process to generate the second filtering result of the grayscale mapping table; S105. The second filtering result is subjected to time-domain filtering to achieve the flickering processing of the infrared image.

[0030] In this embodiment, the average value of the baffle is generated based on the number of rows, columns, and pixel values ​​of the baffle image. , represented as:

[0031] in, The row number of the baffle image. The column number of the baffle image. This indicates the number of samples collected when the baffle is closed. Line number Column pixel values.

[0032] It should be noted that the purpose of the N frames in single-point correction is to calculate the mean. The histogram operation is performed on the images in the normal non-block closed state. In addition, in this embodiment, the block mean is recalculated and updated each time single-point correction is performed, and the block mean obtained from the previous single-point correction is used for calculation between two single-point corrections.

[0033] After obtaining the mean, histogram filtering and grayscale table filtering in AGC are implemented through the FPGA engineering process. This process is described in detail in steps S102 to S104, as follows: The statistical histogram of the (N+1)th frame image is generated by performing length and center position constraints on the histogram based on the mean value of the baffles. , represented as:

[0034]

[0035] in, To restrict the center position of the processed histogram, To limit the length of the processed histogram, This represents the raw pixel values ​​of the infrared image of the actual scene captured with the baffle open.

[0036] The statistical histogram is subjected to a first filtering process to generate the first filtered result of the histogram, which is expressed as follows:

[0037] The result of calculating the statistical histogram for the Nth frame.

[0038] After completing the histogram filtering, update the statistical histogram filtering results to... It is used iteratively during computation:

[0039] After accumulating the first filtering result, a grayscale mapping table is calculated and a second filtering process is performed to generate a second filtering result of the grayscale mapping table, specifically including: The cumulative calculation of the first filtering result is performed according to the following formula to obtain the cumulative calculation result. , represented as:

[0040] A grayscale mapping table is calculated based on cumulative calculation results, preset image contrast, and preset image brightness. , represented as:

[0041] The grayscale map is subjected to a second filtering process to generate the second filtering result of the grayscale map, which is represented as follows:

[0042] in, Indicates the preset image contrast. Indicates the preset image brightness. This represents the calculation result of the grayscale mapping table.

[0043] After completing the grayscale map filtering, update the grayscale map filtering results to... It is used iteratively during computation.

[0044]

[0045] It should be noted that this embodiment employs a two-stage temporal filtering algorithm. This means that a temporal filtering algorithm is first performed at the beginning of the data source, applying temporal filtering to the raw, high-dynamic-range infrared image data. After the AGC and detail enhancement algorithms for image dynamic range conversion are completed, a second temporal filtering is performed to address any remaining temporal noise. This two-stage temporal filtering can more effectively suppress image noise and reduce image flicker. In this embodiment, it is implemented using a temporal denoising model, which specifically includes:

[0046]

[0047] This represents the array of time-domain noise reduction thresholds. This represents the image data from the previous frame. This represents the image data of the current frame. This represents the image data after time-domain filtering. This represents the absolute value of the difference between the current frame and the previous frame's image data. .

[0048] It should be noted that this embodiment performs two levels of temporal filtering. The first level is for the original image data, where the previous and current frame image data are both original image data. The result of the first level of temporal filtering is the image data before the histogram algorithm is performed. The second level of temporal filtering is performed on the image before it is displayed after the enhancement and noise reduction algorithms are completed. At this time, the previous and current frame image data are both enhanced and noise-reduced image data. The result of the second level of temporal filtering is the image provided to the subsequent display module. Used to control the time-domain filtering noise reduction level, the time-domain filtering threshold setting for the second level can be smaller than that for the first level.

[0049] To achieve the above objectives, the FPGA solution used in the specific implementation is the Ti60F100 ultra-small 60K FPGA, which has unique advantages in low power consumption, small size, and high performance FPGAs. The Ti60F100 has 60K logic cells, high-speed I / O configurable for various protocols, and built-in SPI Flash and Hyper-RAM, all contained in a tiny 5.5*5.5mm package with only a 0.5mm pitch, solving the problems of product size and FPGA performance.

[0050] In this embodiment, a true dual-port RAM is first instantiated to store the filtered grayscale map. Then, another true dual-port RAM is instantiated to store the grayscale map for the current frame. During frame intervals, the following steps are performed in three different state machines: calculating the grayscale map for the current frame based on the cumulative histogram, supplementing the data at both ends of the current frame's grayscale map, and writing the filtered grayscale map. The data to be written is selected based on the vsync_cnt count. Simultaneously, during the state machine for writing the filtered grayscale map, the values ​​of the two dual-port RAMs are read, the weighted result is calculated, and the weighted result is written back to the filtered grayscale map RAM. When the next frame's data row is valid, the image data is used as the read address of the two RAMs, and the values ​​of the two dual-port RAMs are read to obtain the data for the filtering switch, which can be used to control the output.

[0051] In practice, the mean processing result from the front end is obtained and synchronized with the current frame from the back end. The image data from the pipeline is transformed in bit width and buffered in a FIFO. When the bus is idle, it is written to DDR buffer area B. Data is read from DDR buffer area A, buffered in the FIFO, and its bit width is transformed. A ping-pong operation is used for buffer space read / write, utilizing the FIFO's empty / full status signal for bus read / write scheduling and control. The current frame mean and current frame image data are used to calculate the current frame's histogram and grayscale map. The current frame mean is then converted to the previous frame mean. The previous frame mean and previous frame image data are used to index the grayscale map.

[0052] Figure 2 An infrared image flicker processing system 200 is provided as an embodiment, comprising: The generation module 201 is used to generate the average value of the baffle based on the baffle image data of N consecutive frames obtained during single-point correction. The flicker processing module 202 is used to process the histogram of the image between two single-point corrections based on the mean value of the baffle to generate a statistical histogram of the image, perform a first filtering process on the statistical histogram to generate a first filtering result of the histogram, accumulate and calculate the first filtering result, calculate the gray-level mapping table and perform a second filtering process to generate a second filtering result of the gray-level mapping table, and perform time-domain filtering process on the second filtering result to realize flicker processing of the infrared image. Display module 203 is used to display the image processed by the flickering processing module in real time.

[0053] This embodiment also discloses an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the infrared image flashing processing method.

[0054] Specifically, the electronic device includes a processor and a memory connected via a system bus. The processor provides computing and control capabilities to support the operation of the entire electronic device. The memory may include non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The computer programs can be executed by the processor to implement an enhanced method provided in the following embodiments. The internal memory provides a cached runtime environment for the operating system computer programs in the non-volatile storage media.

[0055] The various modules in the infrared image flickering processing system provided in this application embodiment can be implemented in the form of a computer program. This computer program can run on a terminal or server. The program modules constituted by this computer program can be stored in the memory of the terminal or server. When the computer program is executed by a processor, it implements the steps of the method described in this application embodiment.

[0056] This embodiment also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, cause the processors to perform the steps of an infrared image flashing processing method. A computer program product containing instructions, which, when run on a computer, causes the computer to perform an infrared image flashing processing method.

[0057] Any references to memory, storage, database, or other media used in the embodiments of this application may include non-volatile and / or volatile memory. Suitable non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which is used as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0058] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0059] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method of infrared image flicker processing, characterized by, Specifically comprising the following steps: Generate the shutter mean value based on the shutter image data of N continuous frames obtained during single-point correction; Generate the statistical histogram of the image after processing the histogram of the image between two single-point corrections according to the shutter mean value; Generate the first filtering result of the histogram by performing the first filtering processing on the statistical histogram; After the accumulation calculation of the first filtering result, calculate the gray mapping table and perform the second filtering processing to generate the second filtering result of the gray mapping table; Realize the flicker processing of the infrared image after the time domain filtering processing of the second filtering result.

2. The infrared image flicker processing method of claim 1, wherein, The shutter mean value is generated based on shutter image data of N continuous frames obtained during single-point correction, specifically including generating the shutter mean value according to the number of rows, the number of columns and pixel point values of the shutter image , is expressed as: , wherein, is the number of rows of the shutter image, is the number of columns of the shutter image, represents the value of the pixel point at the row and the column of the shutter image acquired in the shutter closed state.

3. The infrared image flicker processing method of claim 2, wherein, The step of generating the shutter mean value based on the shutter image data of N continuous frames obtained during single-point correction further comprises: recalculating and updating the shutter mean value during each single-point correction, and using the shutter mean value calculated during the last single-point correction to calculate between two single-point corrections.

4. The infrared image flicker processing method according to claim 2 or 3, characterized by, According to the baffle mean value, a statistical histogram of the image is generated after histogram processing, and the histogram processing specifically includes: generating a statistical histogram of the image after length and center position based restriction processing of the histogram according to the baffle mean value is represented as: ， wherein, to limit the center position of the processed histogram, to limit the length of the processed histogram, represents the raw pixel value of the infrared image of the actual scene collected in the shutter opening state.

5. The method of claim 4, wherein, Generate the first filtering result of the histogram by performing the first filtering processing on the statistical histogram, and the first filtering result of the histogram is represented as: ， The result of the calculation of the histogram for the previous frame of the current image.

6. The infrared image flicker processing method of claim 5, wherein, After the accumulation calculation of the first filtering result, calculate the gray mapping table and perform the second filtering processing to generate the second filtering result of the gray mapping table, and the second filtering processing of the gray mapping table specifically comprises: The accumulated calculation of the first filtering result is performed according to the following formula, to obtain an accumulated calculation result is expressed as: ， Based on the cumulative calculation results, the preset image contrast, and the preset image brightness, a gray scale mapping table is calculated is expressed as: ， After the accumulation calculation of the first filtering result, calculate the gray mapping table and perform the second filtering processing to generate the second filtering result of the gray mapping table, and the second filtering processing of the gray mapping table specifically comprises: ， wherein, represents a preset image contrast, represents a preset image brightness, represents a calculation result of a previous frame image grayscale mapping table of the current image.

7. The method of claim 1, wherein, The time domain filtering processing is performed through a time domain noise reduction model, and the time domain noise reduction model specifically is: ， wherein, denotes a time domain noise reduction threshold array, denotes previous frame image data, denotes current frame image data, denotes time domain filtered image data, denotes an absolute value of a difference between current frame and previous frame image data.

8. An infrared image flicker processing system for implementing the infrared image flicker processing method according to any one of claims 1 to 7, characterized by, The infrared image flicker processing system comprises: A generating module configured to generate the shutter mean value based on the shutter image data of N continuous frames obtained during single-point correction; A flicker processing module configured to generate the statistical histogram of the image after processing the histogram of the image between two single-point corrections according to the shutter mean value, generate the first filtering result of the histogram by performing the first filtering processing on the statistical histogram, after the accumulation calculation of the first filtering result, calculate the gray mapping table and perform the second filtering processing to generate the second filtering result of the gray mapping table, and realize the flicker processing of the infrared image after the time domain filtering processing of the second filtering result; A display module configured to display the image processed by the flicker processing module in real time.

9. An electronic device comprising a memory and a processor, said memory having stored thereon a computer program, characterized in that, The processor executes the computer program to realize the method in any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the method in any one of claims 1-7.

Images